Skip to content
elevatedevco

Cheat sheet · LLMs & Agents

Fine-tuning (LoRA/PEFT) cheat sheet

The key ideas on one page — plus common pitfalls and interview tips. Skim it before an interview, or print it for revision.

Cheat sheet

Parameter-Efficient Fine-Tuning (PEFT) adapts large pretrained models by training a tiny fraction of weights instead of all of them. LoRA and its quantized variant QLoRA are the dominant techniques, slashing GPU memory and storage while matching full fine-tuning quality on most downstream tasks. As of 2026 the standard stack is Hugging Face PEFT + TRL for training and vLLM multi-LoRA for serving.

Why PEFT (vs Full Fine-tuning)

  • Full FT updates all N params: with Adam in mixed precision you store fp32 weights copy + gradients + two optimizer moments (m, v) ≈ 16-18 bytes/param, infeasible for 7B+ on consumer GPUs.
  • PEFT freezes the base model and trains a small set of added/selected params (often <1%), so optimizer state and gradients exist only for the adapters.
  • Benefits: 10-1000x fewer trainable params, tiny checkpoints (MBs not GBs), faster training, easy multi-task serving (one base + many adapters), and reduced catastrophic forgetting since base weights stay frozen.
  • Trade-off: for large distribution shifts (new language, new modality, continued pretraining) full FT or higher-rank adapters may still win.
  • Rule of thumb: prefer prompting/RAG first; fine-tune to teach style/format/behavior or distill a task, not to inject large amounts of new factual knowledge.

LoRA Mechanics

  • Idea: freeze pretrained weight W0 and learn a low-rank update ΔW = B·A, so the adapted weight is W = W0 + (α/r)·B·A.
  • A ∈ R^(r×k) gets a small random init (Kaiming/Gaussian); B ∈ R^(d×r) is init to 0, so ΔW=0 at start and training begins exactly at the pretrained model.
  • Rank r (e.g. 4-64) controls capacity; α is a scaling constant, so the effective scale is α/r. Common practice: set α = r or 2r so increasing r doesn't blow up the effective update magnitude.
  • Only A and B are trained; the d×k base matrix W0 is never updated, which is the source of the memory savings.
  • Adapters are typically applied to attention projections (q,k,v,o); applying to MLP / all linear layers raises capacity at more cost.
  • Inference: ΔW can be merged into W0 (W0 + (α/r)BA) for zero added latency, OR kept separate to hot-swap adapters per request.
  • Params trained per layer ≈ r·(d+k) vs d·k for full — e.g. d=k=4096, r=8 → ~65.5K vs 16.7M (~256x fewer).

QLoRA and Quantization

  • QLoRA = quantize the frozen base to 4-bit (NF4) and train LoRA adapters in higher precision (bf16) on top — enables ~65B fine-tune on a single 48GB GPU.
  • Three innovations: NF4 (4-bit NormalFloat, near-optimal for normally-distributed weights), double quantization (quantize the quantization constants to save more), and paged optimizers (page optimizer state to CPU RAM on memory spikes to avoid OOM).
  • Forward pass dequantizes 4-bit weights to bf16 on the fly for matmul; gradients flow only into the adapters, so memory is dominated by the static 4-bit base.
  • Quality is typically within noise of 16-bit LoRA; the small quantization error is absorbed by the trainable adapters.
  • Watch: merging QLoRA adapters back into a 4-bit base is lossy — merge into a dequantized/16-bit base, or serve the adapter separately.

Hugging Face PEFT in Practice

  • PEFT wraps any HF model: define LoraConfig, call get_peft_model, train with TRL's SFTTrainer; save_pretrained writes only the adapters (MBs).
  • target_modules selects which linear layers get adapters ('all-linear' is a strong default); lora_dropout regularizes; task_type sets the head (CAUSAL_LM, SEQ_CLS).
  • Serving: load base once, then load_adapter / set_adapter to swap adapters for multi-tenant inference; vLLM supports multi-LoRA serving with per-request adapters.
  • merge_and_unload() folds adapters into the base weights, producing a standalone deployable model with no PEFT dependency.
  • Same API covers other methods: prefix/prompt tuning, (IA)^3, AdaLoRA (allocates rank by importance), and DoRA.

Hyperparameters and Variants

  • Rank r: start 8-16; raise to 32-64 for harder/more-divergent tasks. Higher r is not always better and can overfit on small data.
  • Learning rate: LoRA tolerates higher LR than full FT — typically 1e-4 to 3e-4 with cosine decay and warmup; full FT uses ~1e-5 to 5e-5.
  • Epochs: 1-3 on instruction data; more epochs overfit fast. Use early stopping on held-out eval loss.
  • DoRA (weight-decomposed LoRA) splits the update into magnitude + direction for accuracy closer to full FT; rsLoRA rescales by α/√r to stabilize high ranks.
  • Prefer bf16 over fp16 for stability; enable gradient checkpointing and sequence packing to cut memory and padding waste.
  • Alignment beyond SFT: DPO/ORPO/KTO (preference optimization) are commonly run as LoRA fine-tunes on top of an SFT model.

Data, Eval, and Deployment

  • Data quality > quantity: a few thousand clean, diverse, correctly-formatted examples often beat hundreds of thousands of noisy ones.
  • Match the exact chat template and special tokens of the target model; train on completions only (mask the prompt loss) so the model learns to answer, not to echo prompts.
  • Evaluate on a held-out set with task metrics + LLM-as-judge/human review; track regression on general capabilities (fine-tuned models can get narrower).
  • Version each adapter with its base-model hash and tokenizer; an adapter is only valid for the exact base it was trained on.
  • Cost lever: one frozen base + N adapters enables cheap per-customer/per-task models and batched multi-adapter inference.
Read the full lesson