Cheat sheet · Statistics & Probability

Probability Distributions cheat sheet

The key ideas on one page — plus common pitfalls and interview tips. Skim it before an interview, or print it for revision.

A dense, interview-ready reference covering the core discrete and continuous distributions, their parameters, moments, and relationships, plus the practical machinery (MLE, MAP, CLT, sampling, goodness-of-fit) that data scientists and ML engineers are expected to wield fluently.

Foundations: PMF/PDF, CDF, Moments

Discrete RVs have a PMF P(X=x) that sums to 1; continuous RVs have a PDF f(x) where P(X=x)=0 and probability equals area under the curve (an integral over an interval).
CDF F(x)=P(X<=x) always exists (non-decreasing, right-continuous, goes 0->1); for continuous RVs f(x)=F'(x). The inverse CDF (quantile function) drives inverse-transform sampling.
Expectation E[X]=sum x*p(x) (discrete) or integral x*f(x)dx (continuous); linearity E[aX+bY+c]=aE[X]+bE[Y]+c always holds, even under dependence.
Variance Var(X)=E[X^2]-E[X]^2 >= 0; Var(aX+b)=a^2*Var(X). Standard deviation shares the units of X.
Higher moments: skewness measures asymmetry; kurtosis measures tail heaviness. Excess kurtosis = kurtosis - 3 (0 for the Normal).
Var(X+Y)=Var(X)+Var(Y)+2Cov(X,Y); the cross term vanishes only when uncorrelated. Independence implies uncorrelated, but not the reverse (except jointly Normal).

Key Discrete Distributions

Bernoulli(p): single trial; mean p, variance p(1-p) (maximized at p=0.5). Building block for binary labels.
Binomial(n,p): number of successes in n iid Bernoulli trials; mean np, variance np(1-p). Approx Normal for large n; approx Poisson when n is large and p is small.
Poisson(lambda): counts of independent events at constant rate in a fixed interval; mean = variance = lambda. Overdispersion (variance > mean) means switch to Negative Binomial.
Geometric(p): number of trials until the first success (support 1,2,...); mean 1/p, variance (1-p)/p^2. The only memoryless discrete distribution. (Some libraries use the 'failures before first success' version with mean (1-p)/p.)
Negative Binomial(r,p): number of trials/failures until r successes; models overdispersed counts (equivalent to a Poisson with a Gamma-distributed rate).
Categorical/Multinomial: generalize Bernoulli/Binomial to k outcomes; underpin softmax outputs and cross-entropy loss.

Key Continuous Distributions

Normal(mu, sigma^2): symmetric, fully specified by mean and variance; 68-95-99.7 rule. Justified by the CLT; basis of z-scores and many ML assumptions.
Uniform(a,b): constant density 1/(b-a); mean (a+b)/2, variance (b-a)^2/12. U(0,1) is the source for inverse-transform sampling.
Exponential(lambda): waiting time between Poisson events; memoryless; mean 1/lambda, variance 1/lambda^2. Continuous analog of the Geometric.
Gamma(k, theta): positive, right-skewed wait times / sums of exponentials; mean k*theta. Beta(a,b) on [0,1] models proportions/probabilities; mean a/(a+b); conjugate prior for Bernoulli/Binomial.
Log-Normal: log(X) is Normal; right-skewed, strictly positive — incomes, response times, multiplicative processes.
Student's t: heavier tails than Normal, governed by degrees of freedom (df); used for small-sample inference and robust modeling. t with df=1 is the Cauchy (undefined mean and variance); t -> Normal as df -> infinity.

CLT, LLN & Distribution Relationships

Law of Large Numbers: the sample mean converges to E[X] as n grows (justifies Monte Carlo estimation).
Central Limit Theorem: the sample mean of iid RVs with finite variance is approximately Normal as n -> infinity, regardless of the source distribution; standard error = sigma/sqrt(n).
CLT caveats: requires finite variance (fails for Cauchy / heavy tails), approximately iid samples, and a 'large enough' n — heavily skewed data needs a larger n.
Sums/limits: sum of independent Normals is Normal; sum of independent Poissons is Poisson; sum of k iid Exponentials is Gamma/Erlang; Binomial -> Poisson -> Normal under the right limits.
Conjugacy (Bayesian): Beta-Binomial, Gamma-Poisson, and Normal-Normal give closed-form posteriors — a common interview talking point.
Exponential family (Normal, Poisson, Bernoulli, Gamma, etc.) unifies GLMs, yields sufficient statistics, and defines canonical link functions.

Fitting, Sampling & Goodness-of-Fit

MLE: maximize the (log-)likelihood. For the Normal, the MLE mean = sample mean and the MLE variance divides by n (biased); the unbiased sample variance divides by n-1 (Bessel's correction).
MAP adds a prior to MLE (= regularization); an L2 penalty corresponds to a Gaussian prior, an L1 penalty to a Laplace prior.
Sampling: inverse-transform via F^-1(U); rejection sampling; or library RNGs. Use a seeded modern generator (numpy default_rng) for reproducibility.
Check fit visually with histograms, Q-Q plots, and ECDF overlays before trusting any formal test.
Goodness-of-fit tests: Kolmogorov-Smirnov / Anderson-Darling (continuous), Chi-square (binned/categorical), Shapiro-Wilk (normality, small n).
Practical Python: scipy.stats.<dist>.fit(data) returns MLE params; .pdf/.cdf/.ppf/.rvs give density, CDF, quantiles, and samples.

Common pitfalls

Confusing PDF with probability — a density f(x) can exceed 1 and P(X=x)=0 for continuous RVs; only integrals over intervals are probabilities.
Assuming Normality by default; many real quantities (latency, income, counts) are skewed or heavy-tailed — verify with Q-Q plots, not just mean/std.
Forgetting the CLT's finite-variance and iid assumptions; applying it to Cauchy/power-law data or strongly dependent samples gives garbage.
Using Poisson when data are overdispersed (variance >> mean) — switch to Negative Binomial; also watch for zero-inflation.
Mixing up /n vs /(n-1): the MLE variance divides by n (biased), the unbiased estimator divides by n-1.
Treating uncorrelated as independent — zero correlation only rules out linear dependence (the exception is jointly Normal).
Running a KS/Anderson-Darling test with parameters estimated from the same data — this invalidates the standard test (inflated p-values); use Lilliefors or a parametric bootstrap instead.
Over-reading a single goodness-of-fit p-value; large samples reject trivial deviations, small samples lack power.
Sampling without a fixed seed, then being unable to reproduce results.
Confusing parameterizations (Exponential rate lambda vs scale 1/lambda; Gamma shape-scale vs shape-rate; Geometric trials vs failures) — libraries differ, so check the docs.

Interview tips

State the mean and variance of the core distributions from memory (Bernoulli, Binomial, Poisson, Geometric, Normal, Exponential, Uniform).
Map a described scenario to a distribution (e.g., 'count of clicks per hour' -> Poisson; 'time until failure' -> Exponential; 'proportion in [0,1]' -> Beta).
Articulate the CLT precisely, including its assumptions and why the standard error shrinks like 1/sqrt(n).
Walk through deriving an MLE (Bernoulli p-hat = sample mean, or Normal mu/sigma) and connect MAP to regularization.
Discuss the Poisson-vs-Negative-Binomial overdispersion decision, with zero-inflation as a natural follow-up.
Know distribution relationships (Bernoulli->Binomial->Poisson->Normal; Exponential->Gamma; Beta-Binomial conjugacy) — interviewers chain these.
Connect distributions to ML: softmax<->Categorical, cross-entropy<->negative log-likelihood, Gaussian/Laplace priors<->L2/L1, GLMs<->exponential family.
Explain how you'd validate a distributional assumption in practice (Q-Q plot + Anderson-Darling, residual diagnostics) and why fitting then testing on the same data is a trap.

Read the full lesson