Feed-forward network (MLP)
Beginner
The part of each block where every word piece is worked on by itself, with no looking at others.
Novice
Two layers applied to each token separately: widen the vector (usually to 4× its size), apply a nonlinear function, and shrink it back. Most of a transformer’s parameters live here.
Expert
with GELU (GPT-2) or a gated variant such as SwiGLU; weights per block, about two-thirds of the block.
See also: Transformer.