Loss masking
Beginner
Grading the model only on the parts it should learn to write, like its own replies, not the questions.
Novice
Choosing which tokens count in the training loss. In chat fine-tuning, usually only the assistant’s replies count, so the model learns to answer, not to write the user’s side.
Expert
Setting the label of excluded positions to the ignore index (-100 in PyTorch and TRL) so cross-entropy skips them. Assistant-only loss keeps the reply and its end-of-turn token; completion-only loss does the same for prompt–completion data.
See also: Chat template, Cross-entropy loss.