Parameter-efficient fine-tuning (PEFT)
Beginner
Teaching a model a new job by training a small set of extra numbers, instead of changing all of them.
Novice
Methods that train only a small number of parameters, often under 1% of the model, while the original weights stay frozen. LoRA is the most common. PEFT is also the name of Hugging Face’s library for these methods.
Expert
A family (LoRA and its variants, adapters, prompt tuning) that freezes the base and trains a small set of new parameters, cutting optimizer and gradient memory to the trainable part and making per-task checkpoints small.
See also: LoRA (low-rank adaptation), QLoRA.