Positional encoding
Beginner
Extra numbers that tell the model where each word sits in the sentence: first, second, third.
Novice
Information about each token’s position, added to its embedding. Without it, attention would treat the text as an unordered bag of tokens.
Expert
Sinusoidal (fixed), learned absolute (GPT-2), or relative schemes; RoPE rotates query and key vectors by a position-dependent angle so their dot product depends on the distance between tokens.