Top-k and top-p sampling
Beginner
Rules that only let a model pick from its most likely next words, so it doesn’t pick silly ones.
Novice
Ways to cut off unlikely tokens before picking one. Top-k keeps the k most likely; top-p (nucleus sampling) keeps the smallest set whose probabilities add up to p.
Expert
Truncate, renormalize, sample. Top-p adapts the number of candidates to how confident the model is; top-k does not.
See also: Temperature.