Cheat sheet · LLMs & Agents

LLM Fundamentals cheat sheet

The key ideas on one page — plus common pitfalls and interview tips. Skim it before an interview, or print it for revision.

A dense, interview-grade reference on how modern large language models work end-to-end: the decoder-only Transformer, tokenization, training stages (pretraining, SFT, RLHF/DPO), decoding/sampling, scaling laws, and the inference/serving optimizations (KV cache, attention variants, quantization) that dominate real-world deployment in 2026.

Architecture: Decoder-Only Transformer

Modern LLMs (GPT, Llama, Claude, Mistral, Qwen, DeepSeek) are decoder-only and autoregressive: they model p(x) = product of p(x_t | x_<t) and predict the next token via causal (masked) self-attention, so position t only attends to positions <= t.
Self-attention: each token emits Query/Key/Value vectors; attention weights = softmax(QK^T / sqrt(d_k)); the 1/sqrt(d_k) scaling counteracts the O(d_k) growth of dot-product variance, keeping logits out of softmax's saturated, low-gradient regime.
Multi-head attention runs h parallel attention subspaces (each d_k = d_model/h), then concatenates and applies an output projection W_O, letting different heads capture different relations (syntax, coreference, position).
A block = (Pre-LN -> Attention -> residual add) + (Pre-LN -> FFN/MLP -> residual add). Pre-LN (normalize before each sublayer) is standard for stable deep training; RMSNorm (no mean-centering, no bias) commonly replaces LayerNorm for speed.
FFN hidden dim is typically ~4x d_model for ReLU/GELU, or ~8/3 x d_model for gated SwiGLU/GeGLU (to keep param count comparable); gated activations have largely replaced plain ReLU/GELU in recent models. MoE variants replace the dense FFN with sparsely-routed experts (e.g., Mixtral, DeepSeek-V3) to grow total params while keeping active FLOPs per token low.
Positional info must be injected because attention is permutation-invariant: RoPE (rotary embeddings, applied to Q and K) is dominant today for good length behavior, vs older learned/sinusoidal absolute embeddings or ALiBi relative bias.

Tokenization & Embeddings

Text is split into subword tokens via byte-level BPE (GPT, Llama 3) or Unigram/SentencePiece. Byte-level fallback guarantees no out-of-vocabulary token and handles any Unicode, including emoji and rare scripts.
Rule of thumb: ~1 token ~= 0.75 English words ~= 4 characters; code, non-Latin scripts, and rare words tokenize far less efficiently (more tokens per word), inflating cost and context usage.
Each token id maps to a learned embedding vector of size d_model; many models tie the input embedding and output (unembedding) matrices to save parameters.
Context window = max tokens the model can attend over (now 128K-1M+). Cost and naive attention compute scale with sequence length; RoPE scaling / YaRN / NTK-aware interpolation extend usable context beyond the trained length.
Special tokens (BOS/EOS, chat-template role markers like system/user/assistant) structure prompts; using the wrong or missing chat template at inference is a common silent quality bug on open models.

Training Stages: Pretrain -> SFT -> Alignment

Pretraining: self-supervised next-token prediction over trillions of web/code/book tokens; loss = cross-entropy = -(1/N) sum_t log p(x_t | x_<t). This is where almost all knowledge and capability is acquired.
Supervised Fine-Tuning (SFT / instruction tuning): train on curated (prompt, ideal-response) pairs so the base model follows instructions and adopts chat format. Loss is masked to the response (completion) tokens only.
RLHF: train a reward model on human preference pairs (Bradley-Terry objective), then optimize the policy with PPO against that reward minus a per-token KL penalty to the SFT reference (curbs reward hacking and distribution drift).
DPO and variants (IPO, KTO, ORPO, SimPO) skip the explicit reward model and RL loop, optimizing preferences directly with a classification-style loss; simpler, more stable, and now widely used. beta controls deviation from the reference policy.
Perplexity = exp(cross-entropy) measures pretraining fit but NOT helpfulness or safety; alignment quality needs preference/eval benchmarks plus human or LLM-judge ratings.
Parameter-efficient fine-tuning (LoRA/QLoRA) trains small low-rank adapters (W + B@A, with A,B low rank) instead of full weights, cutting trainable params/memory by orders of magnitude and enabling fine-tuning of large models on a single GPU.

Decoding & Sampling

Greedy and beam search maximize likelihood -> deterministic but repetitive and bland for open-ended text; better suited to translation and structured/closed outputs.
Temperature T rescales logits (z/T) before softmax: T<1 sharpens (more deterministic), T>1 flattens (more diverse), and T->0 reduces to greedy.
Top-k keeps the k highest-probability tokens; top-p (nucleus) keeps the smallest set whose cumulative probability >= p, then renormalizes and samples. Top-p adapts to the distribution's shape and is the common default.
Repetition/frequency/presence penalties and min-p further curb loops and tune diversity. For reproducible outputs set temperature 0 (and a fixed seed where supported), but note kernel/batching nondeterminism can still cause minor drift.
Structured outputs: constrained decoding / grammar (JSON schema, regex, CFG) masks invalid tokens at each step to guarantee parseable output, far more reliable than a prompt-only 'return JSON' instruction.

Scaling Laws & Capability Emergence

Test loss falls as a power law in model params N, training tokens D, and compute C; returns diminish smoothly and predictably rather than hitting a sharp wall.
Chinchilla (compute-optimal) result: for a fixed compute budget, scale params and data together (~20 tokens per parameter) rather than over-growing the model and under-training it.
Inference economics push past compute-optimal: smaller models trained on far more tokens (e.g., Llama-style) are intentionally 'over-trained' to be cheaper to serve at a target quality.
Test-time / inference-time scaling (chain-of-thought, self-consistency, and reasoning models that emit long thinking traces, e.g., o-series / DeepSeek-R1) trades more inference compute for accuracy on hard reasoning and math.
Some abilities appear to 'emerge' at scale, but many reported sharp jumps are partly an artifact of discontinuous metrics (exact-match); smoother metrics reveal more gradual gains.
RAG and tool use extend an LLM's effective knowledge and recency without retraining: retrieve relevant chunks via hybrid (vector + lexical/BM25) search and inject them into context, or call functions/APIs.

Inference, Serving & Efficiency

Generation has two phases: prefill (one parallel pass over the full prompt, compute-bound) and decode (one token at a time, memory-bandwidth-bound). Key latency metrics: TTFT (time-to-first-token, ~prefill) and TPOT/inter-token latency (~decode).
KV cache stores past keys/values so each new token costs O(seq) instead of recomputing O(seq^2); cache memory grows linearly with context x batch and is usually the real bottleneck and cost driver in decode.
Attention variants cut KV cost: MQA (1 KV head) and GQA (a few KV heads shared across query heads) shrink the cache; MLA (multi-head latent attention, DeepSeek) compresses KV into a low-rank latent; FlashAttention fuses the kernel for IO-aware exact attention (never materializes the full NxN matrix in HBM).
Quantization (FP8/INT8/INT4, e.g., GPTQ/AWQ) shrinks weights and/or activations for memory and throughput at small quality cost; weight-only INT4 is common for local/edge, and FP8 is increasingly used even for training.
Throughput tricks: continuous (in-flight) batching, PagedAttention for non-fragmented KV memory (vLLM), prefix/prompt caching to reuse shared system prompts, and speculative decoding (a small draft model proposes tokens the big model verifies) to lower latency.
Cost model: you pay per input + output token, and output tokens are pricier and generated serially. Long system prompts, verbose CoT, and large retrieved contexts dominate bills, so cache, trim, and bound output length aggressively.

Common pitfalls

Confusing perplexity/loss with usefulness: low perplexity does not mean helpful, safe, or aligned outputs.
Treating the context window as free memory: info buried mid-context is often missed ('lost in the middle'), and long contexts raise latency and cost.
Forgetting the model is stateless between API calls: 'memory' is just re-sent conversation history (or external storage), not weights that update.
Assuming greedy/temperature-0 is always best: it can be repetitive and still hallucinates; sampling diversity matters for creative and brainstorming tasks.
Using the wrong or missing chat template / special tokens, which silently degrades instruction-following on open models.
Thinking fine-tuning is the default fix: most knowledge/recency gaps are better closed with RAG or tools; fine-tuning mainly changes style, format, and behavior, not facts.
Believing more parameters always wins, ignoring Chinchilla data-scaling and that a smaller, better-trained or better-prompted model can beat a larger one.
Underestimating tokenization effects: code and non-English text consume many more tokens, breaking cost/context estimates and few-shot budgets.
Trusting confident outputs: LLMs hallucinate plausibly and are often poorly calibrated; require citations, verification, or constrained/structured outputs for critical use.
Ignoring that output tokens are generated serially and cost more, so verbose chain-of-thought can dominate latency and price.

Interview tips

Derive scaled dot-product attention from scratch and explain WHY the 1/sqrt(d_k) scaling and the causal mask exist.
Explain the full training pipeline (pretrain -> SFT -> RLHF/DPO) and state what each stage adds and why RLHF uses a KL penalty.
Compare RLHF vs DPO: DPO needs no separate reward model and avoids PPO instability; be ready to write the preference-loss intuition and the role of beta.
Walk through decoding controls (temperature, top-k, top-p, min-p) and recommend settings for deterministic extraction vs creative generation.
Discuss KV cache and prefill (compute-bound) vs decode (memory-bandwidth-bound); mention GQA/MQA/MLA, FlashAttention, and speculative decoding.
Contrast RAG vs fine-tuning vs long-context for adding knowledge, with cost, freshness, and maintenance trade-offs.
Define and distinguish parameters vs context window vs tokens vs embeddings without conflating them.
Explain scaling laws and the Chinchilla compute-optimal result, then note why inference cost motivates over-training smaller models.
Address hallucination and evaluation honestly: how you'd measure quality (benchmarks, LLM-as-judge, human eval) and reduce hallucination (grounding, constrained decoding, verification).
Know 2026 trends: MoE models, reasoning / test-time-compute models, 1M-token contexts, FP8/INT4 quantization, and agentic tool use.

Read the full lesson