Glossary

Top-k and top-p sampling

Beginner

Rules that only let a model pick from its most likely next words, so it doesn’t pick silly ones.

Novice

Ways to cut off unlikely tokens before picking one. Top-k keeps the k most likely; top-p (nucleus sampling) keeps the smallest set whose probabilities add up to p.

Expert

Truncate, renormalize, sample. Top-p adapts the number of candidates to how confident the model is; top-k does not.