KL divergence
Beginner
A number that says how different two sets of guesses are.
Novice
A measure of how far one probability distribution is from another. In training it is used as a leash that stops the model drifting too far from where it started.
Expert
The expected log-ratio of two distributions. RL fine-tuning penalizes KL from a frozen reference policy to limit drift and reward over-optimization; it is estimated per token from samples.