Skip to content
anythingai.space
Training & Adaptationintermediate13 min · 8 steps

LoRA, QLoRA and PEFT

Freeze the giant matrix, train two skinny ones: how LoRA and QLoRA make fine-tuning a 70B model fit on one GPU, and how adapters are served at scale.

GPU memory (70B)1,120GB
trainable100%
70B FULL FINE-TUNE ≈ 1.1 TBWeights · 140 GBGradients · 140 GBOptimizer state · 840 GBOne H10080 GBneed ~14Closed frontierno full fine-tuning
datarequest
metrics are illustrative
1 / 8
ProblemStep 1 of 8

Full fine-tuning a 70B model needs a terabyte

Full fine-tuning keeps weights, gradients and optimizer state for every parameter: roughly 16 bytes each in mixed precision. For 70B parameters that's about 1.1 TB, around fourteen 80 GB GPUs before activations.

Frontier open models are far bigger (Llama 3.1 405B dense, Kimi K3 at 2.8T total), and closed frontier models can't be fully fine-tuned by customers at all.

What changed
  • +Memory for a full 70B fine-tune
  • +One 80 GB GPU for scale

Lesson 3 of 8 in Training & Adaptation