Glossary

Context window

Beginner

How much text a model can look at in one go, like the size of its desk.

Novice

The maximum number of tokens a model can take in at once (prompt plus its own output). GPT-2 had 1,024 tokens and GPT-3 had 2,048; newer models go much further.

Expert

Bounded by the positional scheme it was trained with and by attention cost (quadratic compute, KV cache linear in length). Long-context models extend RoPE and train on long sequences.