Cheat sheet · Statistics & Probability

Statistics Essentials cheat sheet

The key ideas on one page — plus common pitfalls and interview tips. Skim it before an interview, or print it for revision.

A dense, interview-grade reference covering descriptive statistics, probability distributions, estimation, hypothesis testing, regression, and the modern resampling/Bayesian/causal toolkit that data scientists and ML engineers are expected to wield fluently in 2026.

Descriptive Statistics & Data Summaries

Mean is sensitive to outliers; median and trimmed mean are robust. Always report a measure of spread alongside a measure of center.
Variance vs std: std is in the original units. Use n-1 (Bessel's correction) for the sample variance to get an unbiased estimator of the population variance.
Skewness = direction/asymmetry of the tail; kurtosis = tail heaviness (normal has excess kurtosis 0). Right-skew (e.g. income) typically gives mean > median.
Prefer IQR (Q3-Q1) and percentiles for skewed/heavy-tailed data; the empirical 68-95-99.7 rule only holds for approximately normal data.
Covariance is scale-dependent; Pearson correlation normalizes it to [-1, 1]. Use Spearman/Kendall for monotonic but nonlinear relationships.
Correlation (Pearson) measures linear association only — zero correlation does NOT imply independence (e.g. y = x^2 over a symmetric range gives r approximately 0).

Probability & Key Distributions

Bayes: P(A|B) = P(B|A)P(A)/P(B). Posterior is proportional to Likelihood x Prior. Core to spam filters, A/B priors, and Naive Bayes.
Bernoulli/Binomial: discrete success counts. Poisson: counts of rare events over a fixed interval (mean = variance = lambda).
Normal/Gaussian: symmetric, fully defined by mu and sigma; basis of the CLT, z-scores, and most parametric tests.
Exponential: time between Poisson events (memoryless). Uniform: equal density on an interval. Log-normal / power-law: web traffic, wealth, network degrees.
Expectation E[X] = sum(x * p(x)); Variance Var(X) = E[X^2] - (E[X])^2. Linearity of expectation holds even for dependent variables.
Independence: P(A and B) = P(A)P(B). Conditional independence underpins Naive Bayes and probabilistic graphical models.

Sampling, CLT & Estimation

CLT: the distribution of the sample mean approaches normal as n grows, regardless of the population distribution (rule of thumb n >= 30; heavier tails/skew need more).
Standard error of the mean = sigma/sqrt(n); precision improves with sqrt(n), so 4x the data gives ~2x the precision.
Confidence interval: estimate +/- z* * SE. A 95% CI means 95% of such intervals (over repeated sampling) contain the true parameter — NOT a 95% probability for this one interval.
Estimator properties: unbiasedness, consistency (converges as n grows), and efficiency (low variance). Mind the bias-variance tradeoff.
MLE picks parameters that maximize the likelihood of the observed data; it is consistent and asymptotically efficient under regularity conditions.
Use stratified sampling to preserve subgroup proportions; beware selection/survivorship bias that breaks representativeness.

Hypothesis Testing & A/B Tests

Define H0 (no effect) and H1. p-value = P(observing data this extreme or more | H0 true). Reject H0 when p < alpha (commonly 0.05).
Type I error (alpha) = false positive (reject a true H0); Type II error (beta) = false negative. Power = 1 - beta, target >= 0.80.
Test selection: t-test for means (z-test if sigma known/large n), two-proportion z-test for conversion rates, chi-square for categorical association, ANOVA for 3+ group means, Mann-Whitney/Wilcoxon when normality fails.
Power/sample size is driven by effect size, alpha, and variance — compute n BEFORE the experiment; underpowered tests waste traffic.
Multiple comparisons inflate false positives: apply Bonferroni (conservative, controls FWER) or Benjamini-Hochberg (controls FDR).
Statistical significance != practical significance: with huge n, trivial effects become 'significant'. Report effect size and CI, not just p.
Avoid p-hacking and peeking: don't stop a test early on a favorable p-value unless using sequential/always-valid (e.g. mSPRT) methods.

Regression & Statistical Modeling

OLS minimizes the sum of squared residuals; each coefficient estimates the marginal effect of a feature holding the others fixed.
Assumptions (LINE): Linearity, Independence of errors, Normality of residuals, Equal variance (homoscedasticity); also assume low multicollinearity.
R^2 = fraction of variance explained; use adjusted R^2 to compare models with different feature counts since it penalizes complexity.
Logistic regression models the log-odds of a binary outcome: logit(p) = X*beta; exponentiating a coefficient gives an odds ratio.
Detect multicollinearity with VIF (>5-10 is concerning); it inflates coefficient variance and destabilizes interpretation.
Regularization: Ridge (L2) shrinks coefficients; Lasso (L1) drives some to exactly zero (feature selection); both trade a little bias for lower variance.
Check residual plots and influential points (Cook's distance); a high R^2 with patterned residuals signals misspecification.

Resampling, Bayesian & Causal Toolkit

Bootstrap: resample with replacement to estimate the SE/CI of almost any statistic without distributional assumptions — the modern default for messy estimators.
Permutation tests give assumption-light p-values by shuffling labels under H0; great when parametric assumptions are doubtful.
Cross-validation (k-fold) estimates out-of-sample error and is the practical bias-variance diagnostic for ML models.
Bayesian inference yields full posterior distributions and credible intervals (which DO carry a direct probability interpretation), increasingly used in A/B testing.
Causal inference != correlation: prefer randomized experiments; otherwise use difference-in-differences, propensity-score matching, or instrumental variables.
Beware confounders and Simpson's paradox — an aggregate trend can reverse within subgroups; condition on the right covariates, not on colliders.

Common pitfalls

Misinterpreting the p-value as P(H0 is true) or as 'the probability the result is due to chance' — it is conditional on H0 being true.
Treating a 95% confidence interval as a 95% probability that the parameter lies in this specific interval (that is a Bayesian credible interval).
Confusing correlation with causation, and assuming zero correlation implies independence.
Running A/B tests without a pre-computed sample size, then peeking and stopping early (p-hacking) — this inflates false positives.
Ignoring multiple-comparison corrections when testing many metrics or variants simultaneously.
Using mean/std on heavy-tailed or skewed data instead of robust medians/percentiles; outliers silently distort results.
Chasing statistical significance while ignoring effect size — with large n, meaningless differences become 'significant'.
Applying parametric tests (t-test, OLS) without checking assumptions (normality, homoscedasticity, independence).
Falling for Simpson's paradox or survivorship/selection bias by not accounting for confounders or how the data was collected.
Conditioning on a collider or post-treatment variable, which induces spurious associations and biases causal estimates.
Using n instead of n-1 for the sample variance, producing a biased estimate.

Interview tips

Explain the p-value and confidence interval precisely in plain English — sloppy definitions are an instant red flag.
Design an A/B test end to end: hypothesis, primary metric, sample size/power, test choice, multiple-testing correction, and how you would read the result.
Pick the right test from a scenario: means vs proportions vs categorical vs 3+ groups vs non-normal data.
Articulate the bias-variance tradeoff and connect it to regularization, cross-validation, and overfitting.
Walk through Bayes' theorem on a concrete problem (e.g. disease testing at low prevalence) and reason about base rates.
Explain the CLT and why the standard error shrinks with sqrt(n), and when the normal approximation is and isn't usable.
Discuss correlation vs causation and name causal techniques (randomization, diff-in-diff, propensity matching, IV) — increasingly expected in 2026.
Show practical judgment: when you would reach for the bootstrap/permutation tests over parametric formulas, and how you would handle messy real-world data.

Read the full lesson