Glossary

Loss masking

Beginner

Grading the model only on the parts it should learn to write, like its own replies, not the questions.

Novice

Choosing which tokens count in the training loss. In chat fine-tuning, usually only the assistant’s replies count, so the model learns to answer, not to write the user’s side.

Expert

Setting the label of excluded positions to the ignore index (-100 in PyTorch and TRL) so cross-entropy skips them. Assistant-only loss keeps the reply and its end-of-turn token; completion-only loss does the same for prompt–completion data.