LoRA versus full fine-tuning on an 8x8 weight matrix W. Full fine-tuning trains all 64 weights. LoRA freezes W and instead learns a tiny rank-1 update built from a tall 8x1 column A times a wide 1x8 row B, whose product is the same 8x8 shape but uses only 16 trained numbers — 64 versus 16.
LoRA: fine-tune cheaply
Don't relearn the whole matrix — learn a tiny low-rank nudge.
64 trainedvs16 trained
A (8×1) and B (1×8) = 16 numbers build the whole 8×8 update.