Multi-head attention
Beginner
Running several attention steps side by side, each looking for a different kind of clue.
Novice
Attention split into several smaller “heads” that run in parallel, each with its own query, key and value projections, so different heads can track different relationships (meaning, the previous word, and so on).
Expert
d is split into h heads of size d/h; heads are computed independently and concatenated, then projected. The parameter count doesn’t depend on h.