Training & Adaptationintermediate13 min · 8 steps
LoRA, QLoRA and PEFT
Freeze the giant matrix, train two skinny ones: how LoRA and QLoRA make fine-tuning a 70B model fit on one GPU, and how adapters are served at scale.
GPU memory (70B)1,120GB
trainable100%
datarequest
metrics are illustrativeProblemStep 1 of 8
Full fine-tuning a 70B model needs a terabyte
Full fine-tuning keeps weights, gradients and optimizer state for every parameter: roughly 16 bytes each in mixed precision. For 70B parameters that's about 1.1 TB, around fourteen 80 GB GPUs before activations.
Frontier open models are far bigger (Llama 3.1 405B dense, Kimi K3 at 2.8T total), and closed frontier models can't be fully fine-tuned by customers at all.
What changed
- +Memory for a full 70B fine-tune
- +One 80 GB GPU for scale
Lesson 3 of 8 in Training & Adaptation