Cheat sheet · LLMs & Agents

LLM Evaluation & Hallucinations cheat sheet

The key ideas on one page — plus common pitfalls and interview tips. Skim it before an interview, or print it for revision.

A practitioner's guide to measuring LLM quality and reliability: evaluation paradigms (reference-based, LLM-as-judge, human, task benchmarks), the taxonomy and root causes of hallucination, and the detection/mitigation techniques (RAG, grounding, self-consistency, semantic entropy, calibration) expected in 2026 ML interviews.

Evaluation Paradigms & When to Use Them

Reference-based metrics compare output to a gold answer: BLEU/ROUGE/METEOR (n-gram overlap, weak for semantics), BERTScore (token matching over contextual BERT embeddings, reported as precision/recall/F1), exact-match/F1 for extractive QA. Cheap and reproducible but penalize valid paraphrases.
Reference-free / model-based: LLM-as-judge scores or ranks outputs against a rubric; perplexity measures fluency/likelihood only (not correctness). Use when no gold reference exists (open-ended generation, summarization, chat).
Code generation uses execution-based metrics: pass@k = probability that at least one of k sampled solutions passes the unit tests (the unbiased Codex estimator, not just 'k tries'), plus SWE-bench-style repo-level patch resolution.
Human evaluation remains the gold standard for nuance: pairwise preference (A/B), Likert scales, side-by-side. Expensive; needs clear rubrics, multiple raters, and inter-annotator agreement (Cohen's kappa for 2 raters, Fleiss' kappa for 3+, Krippendorff's alpha for missing data).
Capability benchmarks: MMLU-Pro (knowledge), GPQA-Diamond (graduate science), GSM8K/MATH/AIME & FrontierMath (math), HumanEval/MBPP/LiveCodeBench/SWE-bench Verified (code), BBH, IFEval (instruction following), MuSR/ARC-AGI (reasoning), MT-Bench/Arena-Hard & Chatbot Arena Elo (chat preference). Note older sets like MMLU and GSM8K are largely saturated by 2026 frontier models.
Hallucination-specific: TruthfulQA, HaluEval, FELM, FActScore (atomic-fact precision against a knowledge source), SimpleQA (short-form factuality + abstention), SelfCheckGPT (consistency-based).
Match the eval to the deployment: offline benchmark scores rarely predict production quality. Always build a task-specific eval set from real traffic, plus regression and red-team suites.
Beware benchmark saturation and contamination: many public benchmarks leak into pretraining, inflating scores; prefer held-out, freshly authored, or private evals (e.g., LiveBench-style monthly-refreshing sets).

LLM-as-Judge (the dominant scalable method)

Three modes: pointwise scoring (rate 1-5 on a rubric), pairwise comparison (which is better, A or B), and reference-graded (score given a gold answer). Pairwise is more reliable than absolute scores because relative judgments are easier and more stable.
Known biases to control: position bias (favoring the first/second option, fix by swapping order and averaging), verbosity/length bias, self-enhancement bias (a model prefers its own outputs), and sycophancy.
Improve reliability: use a strong judge model, provide explicit rubrics + few-shot anchors, require chain-of-thought reasoning before the verdict, and enforce a structured/JSON output schema.
Validate the judge against humans: measure agreement (Cohen's kappa, Spearman correlation). A judge is only trustworthy if it correlates with human labels on YOUR task.
Use ensembles or a 'jury' of diverse, cheaper judge models to reduce single-model bias; for safety-critical calls keep a human in the loop.
G-Eval pattern: the judge fills a chain-of-thought form, then the score is the probability-weighted expectation over the score tokens (sum of p(s)*s), giving finer granularity than a single argmax integer.

Hallucination: Taxonomy & Root Causes

Definition: fluent, confident output that is unfaithful or factually wrong. Two axes: factuality (contradicts world facts) vs faithfulness/groundedness (contradicts the provided source/context).
Intrinsic hallucination contradicts the input/source; extrinsic adds unverifiable content not in the source. In RAG, faithfulness to retrieved context is usually the key metric.
Root causes: (1) training-data noise/gaps and outdated knowledge; (2) the model is a next-token predictor optimized for plausibility, not truth; (3) exposure bias / autoregressive error accumulation; (4) RLHF can reward confident-sounding answers (sycophancy).
Knowledge cutoff and long-tail facts: models confidently fabricate entities, citations, APIs, and numbers outside their training distribution.
Decoding effects: higher temperature / sampling increases novelty and hallucination risk; greedy or low-temperature decoding is safer for factual tasks.
Recent framing (OpenAI, 'Why Language Models Hallucinate', Sept 2025): hallucinations partly stem from training/eval that reward guessing over saying 'I don't know'; binary 0/1 scoring makes a confident guess strictly better in expectation than abstaining, so better scoring should reward calibrated abstention.
Reasoning models can hallucinate inside the chain-of-thought too, and a plausible CoT does not guarantee a correct final answer (unfaithful reasoning).

Detecting Hallucinations

Consistency-based (SelfCheckGPT): sample N answers at temperature>0; high disagreement across samples signals fabrication. No external knowledge needed (black-box).
Self-consistency for reasoning: sample multiple chains, take a majority vote on the final answer; boosts accuracy and exposes unstable outputs.
Uncertainty signals: token log-probs, sequence entropy, and semantic entropy (Farquhar et al., Nature 2024 — cluster meaning-equivalent samples via bidirectional entailment, then measure entropy over the meaning clusters) flag low-confidence spans.
NLI/entailment checking: verify each generated claim is entailed by the source; FActScore decomposes output into atomic facts and checks each against a knowledge base, reporting the fraction supported.
RAG triad (RAGAS): faithfulness (answer grounded in context), answer/response relevancy, and context precision/recall. Faithfulness = supported claims / total claims.
LLM-as-judge fact-check: prompt a verifier model with each claim + retrieved evidence to label supported / unsupported / contradicted.

Mitigation & Grounding

RAG: retrieve authoritative context and instruct the model to answer only from it and to cite sources; reduces fabrication and adds freshness. Quality is bounded by retrieval recall, chunking, and reranking.
Prompting: explicitly permit 'I don't know', require citations/quotes, ask for step-by-step reasoning, and constrain to provided context. Structured output schemas reduce drift.
Decoding controls: low temperature for factual tasks; constrained/guided (grammar/JSON) decoding for formats; tool use (calculators, code execution, search, SQL) to offload facts the model is bad at.
Self-correction loops: generate, then critique/verify against evidence and revise (e.g., Chain-of-Verification, CoVe); effective but adds latency and can introduce new errors.
Calibration & abstention: train/prompt models to express and respect uncertainty; route low-confidence queries to retrieval, tools, or humans. Confidence thresholds gate auto-responses.
Post-training: fine-tuning / DPO on faithfulness-labeled data and explicitly down-weighting sycophancy; serving-time guardrails (output validators, citation/grounding checkers, PII/safety filters).
No single method fully eliminates hallucination; layer detection + grounding + human oversight proportional to risk.

Production Eval Systems & Metrics

Build a layered eval suite: unit-style assertions (regex / JSON-schema / must-contain), reference scoring, LLM-judge rubrics, and a curated golden set versioned alongside the prompt.
Run evals in CI as regression gates on every prompt/model change; track win-rate vs the current production version, not just absolute scores.
Observability: log prompts, responses, retrieved context, latency, cost, and token usage; sample live traffic for offline grading and to grow the eval set.
Online evaluation: A/B tests, user feedback (thumbs/edits), and guardrail trigger rates measure real impact beyond offline metrics.
Cost/latency are first-class: report quality-per-dollar and tokens-per-request; a cheaper model + good RAG often beats a larger model alone.
Tools to name: RAGAS, DeepEval, OpenAI Evals, LangSmith, Promptfoo, TruLens, HELM, Arize Phoenix, Braintrust, Inspect (UK AISI).
Report metric stability: bootstrap confidence intervals over the eval set and use enough samples; single-run score deltas are noisy and often within CI.

Common pitfalls

Treating BLEU/ROUGE as quality measures for open-ended generation — they reward surface overlap and miss meaning; valid paraphrases score low.
Trusting an LLM judge without validating it against human labels, and ignoring position/verbosity/self-enhancement bias.
Reporting public benchmark scores without checking for data contamination/leakage into pretraining (inflated, non-predictive numbers).
Confusing fluency/confidence with correctness — perplexity and a confident tone say nothing about factuality.
Conflating faithfulness (grounded in provided context) with factuality (true in the world); a RAG answer can be perfectly faithful to a wrong source.
Assuming RAG eliminates hallucination — bad retrieval, poor chunking, or the model ignoring context still produces fabrications.
Using a single eval run and over-interpreting small score deltas without confidence intervals or enough samples.
Optimizing offline benchmarks while ignoring production traffic, cost, latency, and real user feedback.
Evaluating only the final answer for reasoning models while ignoring unfaithful chain-of-thought.
Penalizing 'I don't know' in scoring, which trains/selects models to guess confidently and hallucinate more.

Interview tips

Define hallucination precisely and distinguish intrinsic vs extrinsic and factuality vs faithfulness.
Explain mechanistically why LLMs hallucinate: next-token prediction optimized for plausibility, data gaps/cutoff, RLHF/sycophancy, and decoding temperature.
Walk through designing an eval for a specific product: golden set, LLM-judge with bias controls, regression in CI, then online A/B.
Know LLM-as-judge biases and concrete mitigations: order swapping, rubrics, CoT, pairwise over pointwise, and judge validation via kappa.
Discuss detection in depth: SelfCheckGPT consistency, self-consistency voting, semantic entropy, NLI/FActScore, and RAGAS faithfulness.
Articulate mitigation trade-offs: RAG vs fine-tuning vs tool use vs prompting, and the latency/cost each adds.
Name current benchmarks, explain why they saturate/contaminate, and define 'good' evals (private, refreshing, task-specific).
Show systems thinking: observability, logging retrieved context, quality-per-dollar, guardrails, and human-in-the-loop for high-risk paths.
Cite the 2025 'reward guessing over abstention' result and connect calibrated abstention to lower hallucination rates.
Know pass@k for code eval and why execution-based metrics beat text-overlap metrics for generation tasks.

Read the full lesson