Softmax for Competing Classes

Turn scores into a distribution

For mutually exclusive multi-class classification, the output layer often produces one raw logit per class. Softmax exponentiates and normalizes those logits into positive probabilities that sum to 1. Increasing one class's logit raises its probability relative to every other class.

logits for [database, network, security] = [2.0, 1.0, 0.0]
softmax probabilities                ≈ [0.67, 0.24, 0.09]

Softmax is appropriate when exactly one class is true. In multi-label classification, several labels may be true simultaneously, such as a ticket being both networking and security related; independent sigmoids are a better fit because probabilities need not compete to sum to 1.

Analogy: Softmax divides a fixed pie of confidence among competing choices. Independent sigmoids give each choice its own yes/no meter.

Tip: Match output activation, target encoding, and loss as one design decision. A shape that runs without error can still encode the wrong problem.