GRPO (group relative policy optimization)
Beginner
A way to train a thinking AI: it tries one question many times, and each try is graded against the group’s average.
Novice
A reinforcement-learning method that samples a group of answers to each question and rewards each one by how much better it did than the group average. It needs no separate value model.
Expert
PPO-style clipped updates with the advantage of each sample set to its reward normalized by the group’s mean and standard deviation, plus a KL penalty to a reference model in the loss. Introduced in DeepSeekMath, used for DeepSeek-R1.