Layer normalization
Beginner
A step that evens out a list of numbers, so none grow too big or too small.
Novice
Rescales each token’s vector to have zero mean and unit spread, then applies a learned scale and shift. It keeps the numbers in a steady range as they pass through many layers, which makes training stable.
Expert
Per-token normalization over the feature dimension with gain γ and bias β (2d parameters). Pre-norm placement (before each sub-layer) is standard for deep stacks; RMSNorm drops the mean subtraction.
See also: Residual connection.