Causal inference basics

Correlation measures statistical association between two variables, whereas causation means intervening on one variable changes the distribution of the other. Observational correlation can arise from confounding (a common cause), reverse causation, or selection bias, none of which imply that manipulating the cause would change the effect. Establishing causation requires either a controlled intervention or assumptions (e.g., unconfoundedness) that license a causal reading of the association.

For unit i with potential outcomes Y_i(1) under treatment and Y_i(0) under control, the individual treatment effect is Y_i(1) - Y_i(0). The fundamental problem is that we can only ever observe one of the two potential outcomes for each unit (the one corresponding to the treatment actually received), so the individual effect is never directly observable. Causal inference therefore targets averages (e.g., the ATE) estimable under assumptions like exchangeability.

ATE = E[Y(1) - Y(0)] over the whole population; ATT = E[Y(1) - Y(0) | T=1], the effect among those who actually received treatment; ATC = E[Y(1) - Y(0) | T=0], the effect among the untreated. They coincide when the treatment effect is homogeneous or treatment is randomized, but generally differ when effects are heterogeneous and treatment selection correlates with effect size.

SUTVA (Stable Unit Treatment Value Assumption) requires no interference between units (one unit's treatment does not affect another's outcome) and no hidden versions of treatment (a single well-defined treatment). It is violated under spillovers/network effects, e.g., vaccinating one person reduces infection risk for unvaccinated neighbors (herd immunity), or in marketplace experiments where treating some users affects control users via shared inventory.

A confounder is a variable that causally influences both the treatment and the outcome, inducing a spurious (non-causal) association between them. In a DAG it sits on a backdoor path T <- C -> Y; failing to adjust for it biases the estimated treatment effect. Adjusting for confounders (blocking backdoor paths) is required to identify the causal effect under the backdoor criterion.

Random assignment makes treatment independent of all baseline characteristics, observed and unobserved, so the treated and control groups are exchangeable in expectation. This breaks any backdoor path from treatment to outcome, eliminating confounding, so a simple difference in mean outcomes is an unbiased estimate of the ATE. The key strength is balancing unmeasured confounders, which no observational adjustment method can guarantee.

A collider is a variable that is a common effect of two other variables (A -> C <- B), so the arrows 'collide' at C. The path through a collider is naturally blocked, but conditioning on the collider (or its descendant) opens it, creating a spurious association between A and B. This 'collider bias' (a form of selection bias) is why blindly controlling for everything is dangerous in causal analysis.

A set of variables Z satisfies the backdoor criterion relative to (T, Y) if (1) no node in Z is a descendant of T, and (2) Z blocks every path between T and Y that contains an arrow into T (the backdoor paths). If such a Z exists, the causal effect is identified by adjusting for Z: P(Y | do(T)) = sum_z P(Y | T, z) P(z). It formalizes which covariates remove confounding without introducing collider bias.

do(T=t) represents an intervention that forces T to value t for everyone, severing all incoming arrows to T (graph surgery), and P(Y | do(T=t)) is the resulting outcome distribution. The conditional P(Y | T=t) is mere observation/filtering and reflects how T arose naturally, including confounding. They coincide only when there is no confounding (e.g., under randomization or after valid backdoor adjustment).

The propensity score is e(x) = P(T=1 | X=x), the probability of receiving treatment given covariates. Rosenbaum and Rubin's balancing theorem shows that if treatment assignment is unconfounded given X, it is also unconfounded given the scalar e(X); i.e., conditioning on the propensity score alone suffices to remove confounding. This reduces a high-dimensional adjustment problem to a one-dimensional one.

import numpy as np
from sklearn.linear_model import LogisticRegression
# Estimate propensity scores
ps = LogisticRegression(max_iter=1000).fit(X, T).predict_proba(X)[:, 1]

The IPW (Horvitz-Thompson) estimator is ATE = (1/n) * sum_i [ T_i*Y_i/e(X_i) - (1-T_i)*Y_i/(1-e(X_i)) ], reweighting each unit by the inverse of its probability of the treatment it received to create a pseudo-population with no confounding. It relies on unconfoundedness (correct adjustment set), positivity/overlap (0 < e(x) < 1), a correctly specified propensity model, and SUTVA. Extreme propensities produce high-variance, unstable estimates; in practice the normalized (Hajek/stabilized) variant, which divides by the sum of weights in each arm, is preferred for lower variance.

# Horvitz-Thompson IPW ATE
ate_ht = np.mean(T*Y/ps) - np.mean((1-T)*Y/(1-ps))
# Hajek (normalized/stabilized) variant - lower variance
ate_hajek = np.sum(T*Y/ps)/np.sum(T/ps) - np.sum((1-T)*Y/(1-ps))/np.sum((1-T)/(1-ps))

Positivity requires that every unit has a non-zero probability of receiving each treatment level given its covariates: 0 < P(T=1 | X=x) < 1 for all x with positive density. Without overlap there are covariate regions where one treatment arm is never observed, so the corresponding potential outcome cannot be estimated without extrapolation. Violations cause exploding inverse-probability weights and unstable, model-dependent estimates.

Ignorability states that, conditional on observed covariates X, the potential outcomes are independent of treatment assignment: {Y(0), Y(1)} ⟂ T | X. It means that within strata of X there are no unmeasured confounders, so treated and control units are comparable. It is untestable from data because it concerns unobserved counterfactual outcomes; sensitivity analysis is used to assess robustness to its violation.

An instrument Z affects the treatment T but influences the outcome Y only through T, providing exogenous variation that mimics randomization despite unmeasured confounding. A valid instrument must satisfy: (1) relevance, Z is associated with T; (2) exclusion restriction, Z affects Y only via T; (3) independence/exogeneity, Z is independent of unmeasured confounders (as-good-as-random). Under these (plus monotonicity), IV identifies the Local Average Treatment Effect for compliers.

Stage 1 regresses the endogenous treatment T on the instrument(s) Z (and exogenous covariates) to obtain fitted values T-hat, the part of T explained by exogenous variation. Stage 2 regresses the outcome Y on T-hat to estimate the causal coefficient, purging the confounded variation in T. With a single instrument and treatment under heterogeneous effects, 2SLS estimates the LATE (effect on compliers). Note: weak instruments (low first-stage relevance) bias 2SLS toward the OLS estimate and inflate variance.

DiD compares the change in outcomes over time between a treated group and a control group, estimating the effect as (treated_post - treated_pre) - (control_post - control_pre). This differencing removes time-invariant group differences and common time trends. Its key identifying assumption is parallel trends: absent treatment, the treated group's outcome would have evolved in parallel to the control group's.

RDD applies when treatment is assigned by whether a running variable crosses a threshold (e.g., a test score cutoff for a scholarship). Units just above and just below the cutoff are assumed comparable in all respects except treatment, so the jump (discontinuity) in the outcome at the cutoff identifies a local causal effect. It estimates the effect at the threshold (a local effect) under the assumption that potential outcomes are continuous in the running variable at the cutoff.

Simpson's paradox occurs when an association between two variables reverses (or disappears) after conditioning on a third variable, e.g., a treatment looks beneficial overall but harmful within every subgroup. Whether to aggregate or stratify cannot be decided by data alone; it depends on the causal structure (is the third variable a confounder to adjust for, or a collider/mediator to leave alone?). It demonstrates that causal questions require causal assumptions, not just statistics.

A mediator M lies on the causal path from treatment to outcome (T -> M -> Y), transmitting part of the effect. The total effect is the overall impact of T on Y; the indirect (mediated) effect is the portion flowing through M; the direct effect is the remainder, T -> Y not via M. Conditioning on a mediator estimates (a version of) the direct effect and blocks the indirect effect, so it should NOT be adjusted for when the total causal effect is the target.

Confounders (common causes of T and Y) MUST be adjusted for to remove backdoor confounding. Mediators (on the T->Y path) should NOT be adjusted for when estimating the total effect, since adjusting blocks the indirect effect. Colliders (common effects of two variables) must NOT be adjusted for, as conditioning opens a spurious path (collider/selection bias). The decision depends entirely on each variable's causal role, not its correlation with T or Y.

The frontdoor criterion identifies the effect of T on Y when there is an unobserved confounder of T and Y but a fully mediating variable M is observed such that: M intercepts all directed paths from T to Y, there is no unblocked backdoor path from T to M, and all backdoor paths from M to Y are blocked by T. Then P(Y|do(T)) is recovered by chaining the T->M and M->Y effects, even though direct backdoor adjustment is impossible due to unmeasured confounding.

Rung 1 is Association (seeing): observational queries like P(Y|X), 'what is, given what I observe.' Rung 2 is Intervention (doing): queries like P(Y|do(X)), 'what happens if I act.' Rung 3 is Counterfactuals (imagining): queries like P(Y_x | X', Y'), 'what would have happened had things been different.' Higher rungs cannot generally be answered with only lower-rung information; counterfactuals require a structural causal model.

A counterfactual asks what an individual's outcome would have been under a treatment they did not actually receive, e.g., 'would this patient who took the drug and recovered have recovered anyway?' Answering it requires reasoning about a specific unit's alternative world while holding its latent factors (exogenous noise) fixed. This abduction-action-prediction process needs a structural causal model (functional equations + noise distribution); interventional distributions alone are insufficient.

d-separation is a graphical criterion that determines whether a set of nodes Z blocks all paths between two nodes X and Y, implying conditional independence X ⟂ Y | Z under the DAG's distribution. A path is blocked if it contains a chain or fork with a node in Z (conditioned), or a collider with neither the collider nor its descendants in Z. It links the graph structure to the testable conditional-independence implications of the model.

Matching pairs each treated unit with one or more control units having similar covariates (or similar propensity scores), then compares their outcomes to approximate the counterfactual; averaging over matched pairs estimates the ATT. It is intuitive and non-parametric in the outcome model but suffers from the curse of dimensionality with many covariates (exact matches become rare) and residual imbalance/bias, often requiring bias correction or caliper restrictions. It also relies on unconfoundedness and overlap.

DML uses flexible ML models to estimate nuisance functions (e.g., the outcome regression and the propensity score) and plugs them into a Neyman-orthogonal estimating equation for the target causal parameter. Two properties give robustness: Neyman orthogonality, which makes the estimate first-order insensitive to small nuisance errors, and cross-fitting (sample splitting), which removes overfitting bias. Together they yield root-n consistent, asymptotically normal estimates with valid confidence intervals despite using ML.

The CATE is tau(x) = E[Y(1) - Y(0) | X=x], the average treatment effect for the subpopulation with covariates x, capturing effect heterogeneity. Estimating CATE lets you target interventions at units who benefit most (uplift modeling), rather than relying on a single average effect. Common estimators include meta-learners (S/T/X-learners) and causal forests, all of which require unconfoundedness and overlap.

The S-learner fits a single model with treatment as a feature, mu(x,t), and takes mu(x,1)-mu(x,0); it can shrink the effect toward zero if the model down-weights T. The T-learner fits two separate models (one per arm) and differences them; it is flexible but can be high-variance with imbalanced arms. The X-learner extends the T-learner by imputing individual effects, regressing them, and combining via propensity weights, improving performance when group sizes are very unequal.

Selection bias arises when the sample is selected (or conditioned) on a variable that is a common effect of (or descendant of effects of) treatment and outcome, opening a spurious collider path. Confounding bias arises from a common cause of treatment and outcome via an open backdoor path. The cure differs: confounding is fixed by adjusting for the confounder, whereas selection bias is often worsened by adjustment and instead requires re-weighting or avoiding conditioning on the collider.

A negative-control or placebo test checks an effect that should be zero under the study's assumptions, e.g., a 'treatment' effect on an outcome it cannot causally affect, or an effect in a pre-treatment period. Detecting a non-zero effect signals residual confounding, model misspecification, or a violated assumption (such as parallel trends in DiD). Passing such tests increases credibility but does not prove the main assumptions hold.

LATE is the average treatment effect among 'compliers'—units whose treatment status is shifted by the instrument (they take treatment if encouraged and not if not). It is what IV/2SLS identifies under instrument validity plus monotonicity (no defiers). LATE generally differs from the ATE or ATT because it excludes always-takers and never-takers, so its external validity is limited to the complier subpopulation.

Synthetic control constructs a weighted combination of untreated 'donor' units that closely matches the treated unit's pre-treatment outcome trajectory, then uses this synthetic counterfactual to estimate the post-treatment effect. It is preferred when there is a single (or few) treated unit and many controls, especially when no single control follows parallel trends; the data-driven weights produce a better pre-period fit than an equal-weighted DiD comparison. It is widely used for comparative case studies (e.g., policy effects on one region).

The Augmented IPW (doubly robust) estimator combines an outcome-regression model with inverse-propensity weighting of the residuals. It is doubly robust because it yields a consistent estimate of the ATE if EITHER the propensity model OR the outcome model is correctly specified (not necessarily both). This provides protection against misspecification of one nuisance model and underpins modern methods like DML.

Pre-treatment variables cannot be affected by the treatment, so adjusting for them can only close backdoor paths (reduce confounding) and is generally safe (a caveat is M-bias, where adjusting for a pre-treatment collider can open a path). Post-treatment variables may be mediators (adjusting blocks part of the effect) or colliders/descendants of colliders (adjusting opens spurious paths), so conditioning on them can bias the total effect estimate. A common pitfall is 'controlling away' the very effect you want to measure or inducing collider bias.