MLOps and deployment quiz

Question 1

What is the primary purpose of a model registry in an MLOps pipeline?

Accepted Answer

A store that versions models with metadata, lineage, and stage transitions for controlled promotion

Answer

A cache that stores model predictions to reduce inference latency

Answer

A tool that automatically retrains models on a schedule

Answer

A database optimized for storing raw training data

Question 2

In a canary deployment for an ML model, what is being tested?

Accepted Answer

Routing a small fraction of live traffic to the new model to limit blast radius before full rollout

Answer

Running the new model offline on historical data only

Answer

Sending all traffic to the new model immediately

Answer

Comparing two models purely on a static validation set

Question 3

How does shadow (dark) deployment differ from canary deployment?

Accepted Answer

Shadow logs candidate predictions without serving them; canary serves to a fraction of real users

Answer

Shadow serves all traffic; canary serves none

Answer

They are identical techniques with different names

Answer

Shadow requires a separate cluster while canary cannot

Question 4

What is concept drift, and how does it differ from data (covariate) drift?

Accepted Answer

Concept drift changes P(Y|X); data drift changes the input distribution P(X)

Answer

Concept drift changes P(X); data drift changes P(Y|X)

Answer

They are the same phenomenon

Answer

Concept drift only occurs in unsupervised models

Question 5

Which statistical test is commonly used to detect data drift on a continuous numerical feature?

Accepted Answer

Kolmogorov-Smirnov test

Answer

Chi-squared goodness-of-fit only

Answer

Pearson correlation test

Answer

Shapiro-Wilk normality test

Question 6

What problem does a feature store solve in production ML systems?

Accepted Answer

Consistent, versioned feature definitions shared across training and serving to avoid skew

Answer

Automatic hyperparameter tuning

Answer

Storing model binaries for deployment

Answer

Encrypting model weights at rest

Question 7

What is train/serve skew and what is a common cause?

Accepted Answer

A mismatch between training-time and serving-time feature computation

Answer

Overfitting to the training set

Answer

A GPU memory error during serving

Answer

A version mismatch between Python and the OS

Question 8

When would you choose batch inference over online (real-time) inference?

Accepted Answer

When predictions can be precomputed and low latency is not required

Answer

When sub-10ms latency is mandatory

Answer

When inputs arrive one at a time in real time

Answer

When the model cannot be serialized

Question 9

What is the role of a model serving framework like Triton, TorchServe, or KServe?

Accepted Answer

Standardizes inference APIs and handles batching, GPU scheduling, versioning, and scaling

Answer

Trains models in a distributed cluster

Answer

Labels training data automatically

Answer

Stores experiment metrics

Question 10

What is dynamic (server-side) request batching in model serving and why is it useful?

Accepted Answer

Grouping concurrent requests into one forward pass to boost GPU throughput

Answer

Splitting one request across many GPUs

Answer

Caching identical requests

Answer

Retrying failed requests automatically

Question 11

Why is post-training quantization (e.g., INT8) used in model deployment?

Accepted Answer

Lower precision shrinks size and speeds inference with minor accuracy loss

Answer

It increases model accuracy with no downside

Answer

It is only used during training, never inference

Answer

It eliminates the need for a GPU entirely in all cases

Question 12

What is the difference between horizontal pod autoscaling based on CPU vs. custom metrics for ML serving?

Accepted Answer

Custom metrics like queue depth or GPU util reflect inference load better than CPU

Answer

CPU-based scaling is always optimal for GPU workloads

Answer

Custom metrics cannot be used by Kubernetes HPA

Answer

They produce identical scaling behavior

Question 13

What does DVC (Data Version Control) provide that Git alone does not?

Accepted Answer

Versioning of large data/model files via metafiles plus remote storage

Answer

Faster Python execution

Answer

Automatic model training

Answer

Real-time inference serving

Question 14

In MLflow, what is the difference between the Tracking server and the Model Registry?

Accepted Answer

Tracking logs run params/metrics; Registry manages model versions and stage transitions

Answer

They are the same component

Answer

Tracking serves models; Registry trains them

Answer

Registry logs metrics; Tracking deploys models

Question 15

What is the purpose of a champion/challenger setup in production ML?

Accepted Answer

A current production model evaluated against candidate models on live data

Answer

A method for distributed training

Answer

A data labeling strategy

Answer

A GPU memory optimization technique

Question 16

Why are health checks (readiness vs. liveness probes) important for model serving on Kubernetes?

Accepted Answer

Readiness gates traffic until weights are loaded; liveness restarts hung containers

Answer

Both do the same thing

Answer

Liveness gates traffic; readiness restarts containers

Answer

They are only relevant for batch jobs

Question 17

What is the key advantage of containerizing an ML model with Docker for deployment?

Accepted Answer

Reproducible packaging of model plus dependencies into a portable image

Answer

Automatic accuracy improvement

Answer

Free GPU acceleration

Answer

Built-in data labeling

Question 18

What is a key risk of using `latest` as the image tag for a production model deployment?

Accepted Answer

It is non-deterministic and breaks reproducibility and rollbacks

Answer

It speeds up deployments safely

Answer

It automatically uses the most accurate model

Answer

It encrypts the container

Question 19

In a CI/CD pipeline for ML (CT - continuous training), what triggers an automated retraining run?

Accepted Answer

Schedules, new data thresholds, or drift/performance-degradation signals

Answer

Only manual approval by a human

Answer

A code merge to any branch

Answer

A drop in CPU usage

Question 20

What is the difference between A/B testing and an interleaving experiment for model evaluation?

Accepted Answer

A/B splits users per model; interleaving mixes both models' results within one response

Answer

They are identical

Answer

Interleaving requires no users

Answer

A/B testing is only for offline evaluation

Question 21

What is model lineage and why does it matter for governance?

Accepted Answer

The traceable link from a model to its code, data, params, and environment

Answer

The model's prediction latency

Answer

The number of model versions

Answer

The GPU type used for inference

Question 22

What is the 'cold start' problem in serverless model deployment (e.g., AWS Lambda)?

Accepted Answer

Startup latency from initializing a new instance and loading the model

Answer

The time to label the first data point

Answer

A drift detection failure

Answer

A Kubernetes scheduling bug

Question 23

Why is p99 latency often a more important SLA metric than mean latency for inference services?

Accepted Answer

p99 reflects worst-case tail latency users actually experience, which the mean hides

Answer

Mean is always higher than p99

Answer

p99 measures throughput, not latency

Answer

They always move together so it doesn't matter

Question 24

What is the purpose of a 'rollback' strategy in model deployment, and how is blue-green deployment used for it?

Accepted Answer

Blue-green runs both versions and flips traffic, enabling instant revert to the prior model

Answer

Blue-green deletes the old model immediately

Answer

Rollback requires full retraining

Answer

Blue-green only works for batch jobs

Question 25

In LLMOps, why is prompt versioning important and how does it differ from model versioning?

Accepted Answer

Prompts strongly affect outputs and must be versioned and evaluated independently of model weights

Answer

Prompts never need versioning

Answer

Prompt versioning replaces model versioning

Answer

Prompts are part of the model binary

Question 26

What is KV cache and why does it matter for LLM serving efficiency?

Accepted Answer

Caches past keys/values to avoid recomputing attention, trading large GPU memory for speed

Answer

Caches final predictions to disk

Answer

Stores prompts for versioning

Answer

Compresses model weights to INT8

Question 27

What does continuous batching (in-flight batching) add over static batching for LLM inference?

Accepted Answer

It swaps sequences in/out of the batch per token to keep the GPU saturated

Answer

It disables batching entirely

Answer

It only batches identical prompts

Answer

It is the same as dynamic request batching

Question 28

When deploying a RAG system, what is a common operational reason retrieval quality degrades over time?

Accepted Answer

A stale vector index from un-indexed updates or outdated embedding models

Answer

The LLM weights silently change

Answer

Too many GPUs are allocated

Answer

The prompt template is too short

Question 29

What is the trade-off of an HNSW index vs. a flat (brute-force) index in a production vector database?

Accepted Answer

HNSW is fast/approximate with higher memory; flat is exact but scans everything

Answer

HNSW is always exact and cheaper

Answer

Flat is faster than HNSW at scale

Answer

They give identical latency

Question 30

What is the purpose of input/output validation (e.g., with Pydantic) at a model serving endpoint?

Accepted Answer

Reject malformed inputs early to prevent crashes and garbage predictions

Answer

Speed up GPU inference

Answer

Compress the model

Answer

Version the dataset

Test yourself