Cheat sheet · Data & MLOps

MLOps & Deployment cheat sheet

The key ideas on one page — plus common pitfalls and interview tips. Skim it before an interview, or print it for revision.

A dense, interview-grade reference covering the ML lifecycle in production: packaging and serving models, CI/CD/CT pipelines, feature stores and model/data registries, monitoring for drift and decay, and scalable deployment/inference patterns (including LLMs in 2026).

MLOps Foundations & Lifecycle

MLOps = DevOps + data + models; the core difference is that you version and test THREE artifacts — code, data, and model — not just code.
Google's maturity levels: L0 (manual, notebook-to-prod), L1 (automated training pipeline + Continuous Training), L2 (full CI/CD/CT with automated pipeline deployment).
Add CT (Continuous Training) to CI/CD: pipelines retrain on fresh data triggered by schedule, data volume, or drift/decay alerts.
Reproducibility is the contract: pin code (git SHA), data (snapshot/hash), env (Docker image + lockfile), and seeds so any run is recreatable.
Track lineage end-to-end: which dataset version + code version + hyperparameters produced which model version, and which model serves which endpoint.
Two deployment shapes: batch/offline (scored to a table on a schedule) vs online/real-time (low-latency endpoint); choose by latency SLA and data freshness need.
Note: full determinism is best-effort — GPU non-determinism, library/driver versions, and hardware differences can still cause tiny numerical drift even with fixed seeds.

Packaging, Versioning & Registries

Version EVERYTHING: code in git, data/large artifacts via DVC/LakeFS or object storage, experiments/params/metrics via MLflow or Weights & Biases.
Model Registry (MLflow, SageMaker, Vertex) holds immutable model versions with stages/aliases (e.g. @staging, @prod) and approval gates; MLflow 3 favors aliases over the legacy stage API.
Standardize the artifact: serialize with the framework-native format and ship a signature (input/output schema) + environment so serving is deterministic.
Containerize for parity — the same image runs in dev/CI/prod; avoid 'works on my machine' by baking deps into the image, not the host.
Feature Store (Feast, Tecton, Vertex FS) fights train/serve skew by defining feature logic once and serving it both offline (training) and online (low-latency inference).
Use a model card / dataset card to document intended use, training data, metrics by slice, and limitations — increasingly required for governance/audit (EU AI Act).

CI/CD/CT Pipelines for ML

CI for ML extends code tests with DATA validation (schema, ranges, nulls), model tests (min metric threshold), and behavioral tests (invariance, directional expectations).
CD ships the pipeline AND the model: gate promotion on offline eval + a champion/challenger comparison against the current production model.
CT (Continuous Training) automatically retrains and re-validates; only auto-promote if the challenger beats champion on a held-out set AND passes fairness/latency checks.
Orchestrate DAGs with Airflow, Kubeflow Pipelines, Vertex Pipelines, Dagster, or Prefect; each step is containerized, cached, and idempotent.
Keep a rollback path: registry aliases + immutable images let you flip 'prod' back to the previous version in seconds.
Validate data BEFORE training (Great Expectations / TFDV / Pandera) — bad data silently produces a 'successful' but broken model.

Deployment & Serving Patterns

Online serving: REST/gRPC behind a model server (FastAPI, BentoML, KServe, Triton, TF-Serving, TorchServe); autoscale on QPS/latency, not CPU alone.
Progressive delivery: shadow/dark launch (mirror traffic, don't serve results), canary (small % live), blue-green (instant swap), A/B (statistical comparison of a business KPI).
Latency budgets: optimize p99 (tail), not just the mean; use dynamic request batching, caching, and quantization to hit SLAs.
Optimize inference: quantization (INT8/FP8), distillation, pruning, ONNX/TensorRT compilation, and GPU batching to cut cost-per-call.
Decouple heavy/async work via a queue (endpoint enqueues; workers score) when requests are bursty or long-running.
LLM serving (2026): vLLM/TGI with PagedAttention + continuous (in-flight) batching; KV-cache reuse and speculative decoding raise throughput; serve quantized (AWQ/GPTQ/FP8) weights; tensor/pipeline parallelism for large models.
RAG/LLM apps add deployable components: vector DB, embedding model, prompt/version registry, and guardrails — version prompts like code.

Monitoring, Drift & Observability

Three layers: operational (latency, throughput, error rate, GPU/cost), data quality (schema, missingness), and model quality (metrics, drift).
Data drift = input distribution P(X) shifts; concept drift = the relationship P(y|X) shifts. Concept drift degrades accuracy even when inputs look stable.
Detect feature drift with PSI or the KS-test; PSI > 0.2 signals significant shift. Monitor prediction drift as an early, label-free proxy.
Ground-truth labels are usually delayed — track proxy/leading metrics now and reconcile true accuracy once labels arrive; alert on sustained degradation.
LLM observability: log prompts/responses/tokens/cost, run eval suites (LLM-as-judge, groundedness/faithfulness for RAG), and detect hallucination + prompt injection.
Close the loop: drift/decay alert → trigger retraining (CT) or human review; log every prediction with feature values for debugging and audits.

Infra, Scaling & Governance

Kubernetes is the common substrate; managed platforms (SageMaker, Vertex AI, Databricks, Azure ML) let you avoid rebuilding orchestration/registry/serving.
Separate training (GPU, batch, spot-friendly, fault-tolerant) from serving (right-sized, autoscaled, HA) — different cost and reliability profiles.
IaC (Terraform) + GitOps make environments reproducible and auditable; promote the SAME image dev → staging → prod.
Cost control: spot/preemptible for training, scale-to-zero for spiky endpoints, right-size GPUs, and quantize/distill before buying bigger hardware.
Security & governance: secrets in a vault, PII handling, model/data access controls, audit logs, signed artifacts/supply-chain checks, and lineage for compliance (EU AI Act, model risk).
Define SLOs/SLAs (availability, latency, freshness) and on-call runbooks; treat a model endpoint as a production service, not a notebook.

Common pitfalls

Confusing data drift (P(X) shifts) with concept drift (P(y|X) shifts) — and assuming stable inputs mean a healthy model.
Train/serve skew: computing features differently in training vs serving; a top cause of silent online degradation (a feature store fixes it).
Versioning only code, not data and models — making runs irreproducible and rollbacks impossible.
Deploying without monitoring or a rollback plan, then discovering decay only when a business metric drops.
Optimizing mean latency while p99/tail latency blows the SLA under load.
Auto-promoting a retrained model without a champion/challenger gate, letting a worse model into production.
Data leakage during training (target/temporal leakage) yielding great offline metrics that collapse in production.
Treating LLM apps as stateless code: not versioning prompts, ignoring token/cost monitoring, and skipping hallucination/injection guardrails.
Retraining blindly on a fixed schedule instead of triggering on drift/decay — wasting compute or retraining too late.
Waiting for ground-truth labels to detect problems instead of monitoring prediction drift and proxy metrics.

Interview tips

Be ready to draw an end-to-end architecture: ingestion → feature store → training pipeline → registry → CI/CD → serving → monitoring → retraining loop.
Expect 'how do you detect and respond to drift?' — name PSI/KS, prediction drift as a label-free proxy, and the retraining trigger.
Articulate the difference between CI/CD and CT, and what specifically you test (data, model metrics, behavior) beyond unit tests.
Compare deployment strategies (shadow, canary, blue-green, A/B) and say when you'd use each and how you'd roll back.
Explain how you prevent train/serve skew and why a feature store matters.
Discuss reproducibility concretely: pin code SHA, data hash, env, seeds; describe model registry stages/aliases.
For 2026 LLMOps: discuss vLLM/continuous batching, KV-cache, quantization, RAG evaluation (groundedness/faithfulness), prompt versioning, and guardrails.
Tie technical choices to cost and SLAs — interviewers probe latency budgets, GPU cost optimization, and autoscaling decisions.
Have a real story: a model that degraded in production, how you detected it, root-caused it, and fixed the pipeline.

Read the full lesson