Policy gradient
Beginner
A way to train by trial and error: make the choices that scored well a little more likely next time.
Novice
A family of reinforcement-learning methods that adjust the model so that actions which led to high reward become more probable and those that led to low reward less probable.
Expert
Gradient ascent on expected reward using the score-function estimator: weight each sampled action’s log-probability gradient by its advantage (reward minus a baseline). For LLMs the actions are tokens and the policy is the model.