Glossary

Parameter-efficient fine-tuning (PEFT)

Beginner

Teaching a model a new job by training a small set of extra numbers, instead of changing all of them.

Novice

Methods that train only a small number of parameters, often under 1% of the model, while the original weights stay frozen. LoRA is the most common. PEFT is also the name of Hugging Face’s library for these methods.

Expert

A family (LoRA and its variants, adapters, prompt tuning) that freezes the base and trains a small set of new parameters, cutting optimizer and gradient memory to the trainable part and making per-task checkpoints small.