Softmax
Beginner
A math step that turns a list of scores into chances that add up to 100%.
Novice
A function that turns a list of scores into probabilities: it exponentiates each score and divides by the total, so bigger scores get a bigger share and everything sums to 1.
Expert
. Invariant to adding a constant, so implementations subtract the max for stability. Used for attention weights and for the next-token distribution.
See also: Logits.