What are bias and variance in machine learning, simply?

Bias is systematic error — the model is too simple or wrongly shaped to capture the truth, so it's consistently off. Variance is instability — the model is so flexible it fits the noise of its particular training sample and changes drastically when retrained on different data.

How do I know if my model has high bias or high variance?

Compare training and validation performance. Bad at both → high bias (underfitting). Excellent on training but much worse on validation → high variance (overfitting). The gap between the two scores is the key signal.

Does more training data reduce bias or variance?

Mainly variance — more data makes the fitted model steadier and less at the mercy of any one sample. It does little for bias: a model family too simple for the problem stays wrong no matter how much data you give it.

What is irreducible error?

The noise inherent in the data-generating process itself — measurement error, genuinely random outcomes, missing causal factors. It sets a floor under the error of every possible model, which is why 100% accuracy is usually the sign of a bug, not a breakthrough.

ScienceJuly 15, 2026Learn AI & Data Science

The Bias–Variance Tradeoff, Explained With Intuition (Not Just Formulas)

What bias and variance actually mean, why total error decomposes into bias² + variance + noise, and how the tradeoff guides every modelling decision.

Imagine asking many archers to hit a target. One archer's arrows cluster tightly — but around the wrong spot: that's bias, a systematic error baked into the aim. Another's arrows scatter widely around the right spot: that's variance, sensitivity to every gust of wind. In machine learning the 'gust of wind' is which particular training sample you happened to draw — a high-variance model changes its predictions dramatically when retrained on a slightly different dataset, while a high-bias model gives you the same wrong answer no matter what you train it on.

The decomposition

For squared-error problems, the expected error of a model on a new point splits cleanly into three terms. Bias²: how far the *average* model (over all possible training sets) is from the truth — the error of your model family being too simple or wrongly shaped. Variance: how much individual trained models scatter around that average — the error of being too sensitive to the specific sample. Irreducible noise: randomness in the data itself that no model can predict. Modelling choices move the first two; nothing moves the third.

Complexity slides you along the curve

A straight line fitted to curved data has high bias — it *cannot* represent the truth, so it's systematically wrong everywhere, on training data and test data alike. A 15th-degree polynomial through the same points has low bias but high variance — it can represent almost anything, so it contorts itself around the noise in this particular sample and predicts wildly on the next one. As complexity grows, bias falls and variance rises; total error traces a U-shape, and the art is finding the bottom of the U. In practice you don't compute bias and variance directly — you read their symptoms via cross-validation.

Symptom	Diagnosis	Moves
Poor on training AND validation	High bias (underfitting)	Bigger/more flexible model, better features, less regularization, train longer
Great on training, poor on validation	High variance (overfitting)	More data, regularization, simpler model, early stopping, ensembling
Good on both, gap small	Near the sweet spot	Ship it — or push complexity slightly and re-check

A modern footnote

Very large neural networks trained with strong regularization and huge data can defy the classic U-curve ('double descent'): past a threshold, adding parameters lowers test error again. The classical tradeoff remains the right mental model for most practical problems — just know the frontier is more nuanced.

The tradeoff is the theory behind almost every practical fix you'll ever apply: it's why regularization exists (accept a little bias, cut a lot of variance), why bagging averages many models (variance ↓) while boosting stacks weak ones (bias ↓), and how you diagnose overfitting vs underfitting. Drill it further on the bias–variance tradeoff topic page, or in **AI Learning** — cheat sheets and ~10,000 offline practice questions that take you from this intuition to interview-ready.

AI Learning: AI/ML/DS Q&A

Free · Works offline · Turn intuition into exam- and interview-ready understanding.

Coming soon toGoogle Play

Sources

Hastie, Tibshirani & Friedman — The Elements of Statistical Learning, §2.9 & ch. 7
Geman, Bienenstock & Doursat (1992) — 'Neural Networks and the Bias/Variance Dilemma', Neural Computation
Belkin et al. (2019) — 'Reconciling modern machine-learning practice and the classical bias–variance trade-off', PNAS