Glossary

Overfitting

Beginner

When a model learns its practice examples by heart instead of learning the skill, so it does badly on new ones.

Novice

When a model fits its training examples so closely that it gets worse on new ones. The sign is training loss still falling while loss on held-out examples rises.

Expert

Train/validation divergence from memorization: too many epochs, too high a learning rate or rank for the data size, or low diversity. Early stopping, fewer epochs and deduplicated data are the first fixes.