Cheat sheet · Data & MLOps

Python for Data Science cheat sheet

The key ideas on one page — plus common pitfalls and interview tips. Skim it before an interview, or print it for revision.

A dense reference for the core Python data stack (NumPy, pandas, scikit-learn) plus performance, correctness, and ML-engineering patterns interviewers probe in 2026. Emphasizes vectorization, leakage-free pipelines, and reproducibility.

NumPy: Vectorization & Broadcasting

ndarray is a contiguous, typed, fixed-dtype buffer; operations are vectorized C loops, so avoid Python for-loops over elements (often 10-100x faster).
Broadcasting aligns shapes right-to-left; each dimension must be equal or one of them 1. (m,1) op (1,n) -> (m,n) without materializing a copy.
Basic slicing returns VIEWS (shares memory); fancy/boolean indexing returns COPIES on read. Mutating a view mutates the parent, a classic silent bug.
Prefer ufuncs and preallocated/in-place ops; use axis= correctly (axis=0 collapses rows -> one value per column).
dtype matters: float32 halves memory vs float64; integer overflow is silent (wraps modulo). Use float64 for numerically stable large sums (or math.fsum / np.add.reduce with care).
Use np.einsum for explicit, readable tensor contractions; np.where / np.select for vectorized conditionals.

pandas: Idiomatic, Fast, Correct

Select with .loc (label) and .iloc (position). Never chain assignment like df[df.x>0]['y']=1; it raises SettingWithCopyWarning and may silently no-op. Use one .loc[mask, 'y'] = ... call.
Vectorize: use built-in methods, .map / .assign, and boolean masks. Avoid iterrows and apply(axis=1); they are Python-level loops.
groupby-agg is split-apply-combine; prefer named aggregation: df.groupby('k').agg(n=('v','size'), avg=('v','mean')). transform returns a result broadcast back to the original index.
merge() = SQL join (use validate= and indicator=True to catch fan-out/duplicates); concat() stacks frames; join() merges on the index.
Use category dtype for low-cardinality strings and downcast numerics to cut memory; call pd.to_datetime once, then use the .dt accessor.
Chain with .pipe() + .assign() to keep transforms pure and debuggable. Copy-on-write is opt-in in pandas 2.x and the default in pandas 3.0, which removes most chained-assignment footguns.
For larger-than-RAM data, reach for Polars (lazy, multi-threaded, Arrow-backed) or DuckDB SQL over Parquet.

Data Cleaning & Feature Engineering

Distinguish missingness: MCAR / MAR / MNAR. Imputation (mean/median/KNN/iterative) must be FIT on train only, then applied to val/test.
Encode categoricals: one-hot (low cardinality), target/mean encoding done inside CV folds (high cardinality, leakage-prone), ordinal only when the order is real.
Scale features for distance- and gradient-based models (StandardScaler, MinMaxScaler, RobustScaler for outliers). Tree/boosting models do not need scaling.
Detect outliers via IQR (below Q1-1.5*IQR or above Q3+1.5*IQR) or z-score; decide cap/clip/remove deliberately, not reflexively.
Handle skew with log1p / Box-Cox (positive only) / Yeo-Johnson; build interaction and time-based features (lags, rolling windows) without leaking the future.
Class imbalance: resample on train only (e.g. SMOTE inside the CV pipeline), use class_weight, or tune the decision threshold. Never oversample before the split.

scikit-learn: Leakage-Free Pipelines

Wrap every preprocessing + model step in a Pipeline/ColumnTransformer so transformers .fit only on training folds, the #1 defense against leakage.
Estimator API: fit / transform / predict / predict_proba; fit_transform = fit then transform. Always fit on train, only transform on test.
Validate with cross_val_score / StratifiedKFold (classification), GroupKFold (grouped entities), TimeSeriesSplit (temporal, no shuffling).
Tune with GridSearchCV / RandomizedSearchCV / HalvingGridSearchCV or Optuna; pass the whole pipeline so preprocessing is re-fit on each fold.
Choose metrics for the problem: ROC-AUC vs PR-AUC (imbalance), precision/recall/F1 by threshold, RMSE/MAE/R2 for regression, log-loss/Brier for probabilities.
Calibrate probabilities (CalibratedClassifierCV with Platt/sigmoid or isotonic) when you act on predict_proba; persist models with joblib and pin library versions.

Performance, Memory & Scale

Profile before optimizing: %timeit, cProfile + snakeviz, line_profiler, memory_profiler; fix the actual hotspot, not a guess.
Hierarchy of speedups: vectorize (NumPy/pandas) -> Numba/Cython/numexpr -> multiprocessing for CPU-bound work -> distributed (Dask/Spark/Ray).
The GIL serializes Python bytecode: use processes (or NumPy code that releases the GIL) for CPU-bound work, threads/async for I/O-bound. Free-threaded CPython is experimental in 3.13 and officially supported (still opt-in) from 3.14.
Use generators and chunked reads (read_csv(chunksize=)) plus Arrow/Parquet columnar formats to cut memory and I/O vs CSV.
Prefer columnar engines for big data: Polars and DuckDB run multi-threaded and out-of-core, often beating pandas without a cluster.
Reduce memory: downcast dtypes, use category strings, drop unused columns early, and avoid intermediate copies in long chains.

Reproducibility & ML Engineering Hygiene

Set and propagate seeds / random_state (NumPy Generator, sklearn, framework) so splits, init, and resampling are deterministic.
Pin environments: pyproject.toml + lockfile (uv / Poetry) or conda; record the exact library versions used to train each model.
Track experiments (MLflow / Weights & Biases): params, metrics, artifacts, and data version (DVC) so results are auditable.
Split BEFORE any fitting/scaling/encoding; keep a held-out test set untouched until final evaluation.
Add data-validation/contract tests (pandera / Great Expectations) and unit-test feature functions; assert schema, dtypes, and value ranges.
Reuse the saved pipeline for both train and serve to prevent training-serving skew; monitor data and prediction drift in production.

Common pitfalls

Data leakage: fitting scalers/imputers/encoders or doing feature selection/resampling before the train-test split, or using future information in time series.
Chained indexing in pandas (df[mask]['col'] = x) triggers SettingWithCopyWarning and may silently fail to write; use a single .loc[mask, 'col'] = x.
Confusing NumPy/pandas views vs copies, so mutating a slice unexpectedly changes (or fails to change) the original.
Looping with iterrows/apply(axis=1) instead of vectorizing: orders of magnitude slower and harder to read.
Using accuracy on imbalanced data; ignoring PR-AUC, recall, and threshold tuning when the positive class is rare.
Float precision/overflow bugs: silent integer wraparound, summing many float32 values, and comparing floats with == instead of np.isclose.
Using threads for CPU-bound work: blocked by the GIL; you need processes or vectorized/native code.
Mismatched train/serve preprocessing (training-serving skew) and forgetting to seed RNGs, making results non-reproducible.
Mutable default arguments and global state in feature functions causing subtle, hard-to-trace bugs.
Optimizing without profiling, or reaching for Spark/Dask when Polars/DuckDB or simple vectorization would do.

Interview tips

Expect to explain broadcasting rules and to vectorize a naive Python loop on the spot.
Be ready to spot and fix data leakage in a given workflow and justify the correct CV strategy (Stratified vs Group vs TimeSeries).
Articulate views-vs-copies and SettingWithCopyWarning, a favorite pandas gotcha question.
Justify metric choice for a business scenario (e.g., fraud/imbalance -> PR-AUC + recall + cost-based threshold).
Discuss the GIL and when you would use threads vs processes vs async vs distributed compute.
Walk through scaling a pipeline as data grows from MBs to TBs (chunking -> Polars/DuckDB -> Dask/Spark).
Show end-to-end thinking: reproducibility (seeds, env pinning), experiment tracking, and monitoring drift in production.
Live-coding: clean a messy DataFrame (missing values, dtypes, dedup) and build a leakage-safe sklearn Pipeline.

Read the full lesson