LoRA (low-rank adaptation)
Beginner
A cheap way to retrain a big model: freeze it and train two thin add-on pieces.
Novice
A fine-tuning method that freezes the original weights and learns a small correction for some of them, written as the product of two thin matrices. Far fewer numbers are trained and stored.
Expert
with , , ; B starts at zero. Merged into W after training, so inference costs nothing extra.
See also: Fine-tuning.