ML system design quiz

Question 1

In a large-scale recommendation system, why is a two-stage architecture (candidate generation + ranking) preferred over a single ranking model scoring the entire catalog?

Accepted Answer

It lets a cheap retriever cut millions of items to a few hundred so an expensive ranker scores only those within latency budget

Answer

Single-stage models cannot learn user preferences

Answer

Two stages are required to support A/B testing

Answer

Candidate generation eliminates the need for embeddings

Question 2

What is training-serving skew, and what is the most robust engineering remedy?

Accepted Answer

Reuse identical feature transformation code/feature store across training and serving

Answer

Increase model capacity to absorb the discrepancy

Answer

Retrain the model more frequently

Answer

Add dropout during inference

Question 3

When designing a feature store, what is the key difference between the online and offline stores?

Accepted Answer

Offline = high-throughput historical reads for training; online = low-latency key-value reads for serving

Answer

Offline stores raw data; online stores model weights

Answer

Offline is for streaming; online is for batch

Answer

They store identical data but in different regions

Question 4

Why is point-in-time correctness critical when generating training data from a feature store?

Accepted Answer

It prevents future feature values from leaking into past training rows via as-of joins

Answer

It compresses the training data

Answer

It ensures features are normalized

Answer

It guarantees class balance

Question 5

In an A/B test for a new ranking model, the treatment shows higher CTR but lower long-term retention. How should you design the evaluation?

Accepted Answer

Use long-term guardrail metrics (retention/revenue) with holdbacks, not just CTR

Answer

Ship treatment because CTR improved

Answer

Shorten the test to reduce variance

Answer

Optimize purely for CTR since it is easiest to measure

Question 6

What problem does shadow (dark) deployment solve before fully launching a new model?

Accepted Answer

It runs the new model on live traffic without serving its outputs, to validate safely

Answer

It splits users into control and treatment

Answer

It gradually increases traffic to the new model

Answer

It rolls back automatically on errors

Question 7

What is the difference between data drift and concept drift, and why does it matter for monitoring?

Accepted Answer

Data drift changes P(X); concept drift changes P(Y|X) and degrades accuracy directly

Answer

They are synonyms for the same phenomenon

Answer

Data drift only happens in images; concept drift only in text

Answer

Concept drift changes the input distribution while data drift changes labels

Question 8

For a fraud detection system, labels arrive with weeks of delay (chargebacks). How does this label latency affect system design?

Accepted Answer

You need a label-maturation/embargo window and proxy monitoring since true labels lag

Answer

You can label all recent transactions as legitimate

Answer

Label delay has no impact on training

Answer

You should switch to unsupervised learning entirely

Question 9

Why are approximate nearest neighbor (ANN) indexes like HNSW used in vector search instead of exact KNN?

Accepted Answer

ANN gives sub-linear search with tunable recall vs exact KNN's O(N) scan

Answer

ANN is always more accurate than exact KNN

Answer

Exact KNN cannot handle high dimensions at all

Answer

ANN removes the need for embeddings

Question 10

In a RAG system, retrieved context is relevant but the LLM still hallucinates. What design changes most directly address this?

Accepted Answer

Enforce grounding/citations and add an entailment/verification step with abstention

Answer

Increase temperature for more creative answers

Answer

Remove the retriever entirely

Answer

Only enlarge the context window

Question 11

What is the purpose of a model registry in an MLOps platform?

Accepted Answer

Versioned, governed storage of models with lineage/stages for reproducible deploy and rollback

Answer

A place to store raw training data

Answer

A real-time feature cache

Answer

A dashboard for A/B test results

Question 12

When should you choose batch (offline) prediction over real-time (online) serving?

Accepted Answer

When predictions can be precomputed and don't need request-time inputs or low latency

Answer

Whenever the model is a neural network

Answer

Only when you have GPUs

Answer

When you need sub-10ms latency

Question 13

How do multi-armed bandits differ from classic A/B tests, and when are they preferable?

Accepted Answer

Bandits adaptively reallocate traffic to reduce regret; good for short-lived/many-variant cases

Answer

Bandits always require more traffic than A/B tests

Answer

Bandits cannot explore alternatives

Answer

A/B tests adapt allocation while bandits are fixed

Question 14

What is the cold-start problem in recommender systems and a practical mitigation for new items?

Accepted Answer

Use content/metadata features plus exploration to bootstrap items lacking interaction data

Answer

Ignore new items until they get clicks

Answer

Only recommend the most popular item to everyone

Answer

Disable personalization permanently

Question 15

Why might you decouple feature computation from model inference using a feature pipeline that materializes features asynchronously?

Accepted Answer

Heavy features can't be computed within latency budget, so precompute to online store (trading freshness)

Answer

Async computation improves model accuracy directly

Answer

It removes the need for an online store

Answer

It eliminates feature drift

Question 16

What is the role of a candidate generation embedding model trained with a two-tower (dual-encoder) architecture?

Accepted Answer

Separate user/item encoders share a space so items are precomputed and retrieved via ANN

Answer

A single network that scores every item jointly with the user

Answer

A model that only ranks already-retrieved items

Answer

A rule-based filter for candidates

Question 17

In LLM serving, what is the difference between the prefill and decode phases and why does it affect batching strategy?

Accepted Answer

Prefill is parallel/compute-bound; decode is sequential/memory-bound, motivating continuous batching

Answer

Both phases have an identical compute profile

Answer

Decode is parallel and prefill is sequential

Answer

Batching strategy is irrelevant to LLM serving

Question 18

What is the KV cache in transformer inference and what system constraint does it create?

Accepted Answer

Cached keys/values avoiding recompute; its memory grows with seq len x batch, limiting concurrency

Answer

A cache of model weights to speed loading

Answer

A disk cache for prompts

Answer

A cache of tokenizer outputs only

Question 19

How does an LLM-as-judge evaluation pipeline introduce bias, and how do you mitigate it?

Accepted Answer

Position/verbosity/self bias; mitigate via order-swapping, rubrics, and human calibration

Answer

LLM judges have no bias if temperature is 0

Answer

Bias is fixed only by using the same model as judge and candidate

Answer

Longer prompts always eliminate bias

Question 20

What is the lambda vs kappa architecture distinction in ML data pipelines?

Accepted Answer

Lambda has separate batch+speed layers; kappa unifies on one streaming path via replay

Answer

Kappa uses two layers; lambda uses one

Answer

They are identical architectures

Answer

Lambda is only for images, kappa for text

Question 21

Why is offline metric improvement not sufficient to launch a model, and what bridges the offline-online gap?

Accepted Answer

Offline data reflects the old policy; online A/B (or off-policy eval) is needed to decide

Answer

Offline metrics are always wrong

Answer

Online tests are unnecessary if AUC improves

Answer

There is no gap if the dataset is large

Question 22

What is a feedback loop in a deployed ML system and why is it dangerous?

Accepted Answer

Predictions shape future training data, reinforcing the model's own biases

Answer

It is when users give explicit ratings

Answer

It speeds up convergence harmlessly

Answer

It only affects supervised image models

Question 23

For low-latency serving, what is the benefit and risk of caching model predictions?

Accepted Answer

Cuts latency/compute but risks staleness; key cache by model version with TTL/invalidation

Answer

Caching always improves accuracy

Answer

Caches never go stale for ML

Answer

It removes the need for a model server

Question 24

What is the purpose of canary deployment and how does it differ from blue-green deployment?

Accepted Answer

Canary ramps a small traffic slice gradually; blue-green switches all traffic at once

Answer

Canary needs two full environments; blue-green uses one

Answer

They are the same technique

Answer

Blue-green tests on a small percentage first

Question 25

In designing a real-time feature for 'number of user clicks in the last 5 minutes', what infrastructure pattern fits best?

Accepted Answer

Stream-processing windowed aggregation written to the online store

Answer

A nightly batch job

Answer

An on-demand SQL scan over all raw events per request

Answer

Storing the raw event list inside the model

Question 26

Why is model quantization (e.g., INT8/FP8) used in production serving and what is the main tradeoff?

Accepted Answer

Smaller/faster/cheaper inference at the cost of possible accuracy loss

Answer

It always improves accuracy

Answer

It only helps training speed

Answer

It removes the need for GPUs entirely

Question 27

What is the difference between horizontal and vertical scaling for an ML inference service, and which suits GPU models?

Accepted Answer

Vertical = bigger node; horizontal = more replicas; GPU serving usually scales horizontally

Answer

Horizontal means a bigger single GPU

Answer

GPU models can only scale vertically

Answer

Both terms mean the same thing

Question 28

How would you detect prediction-serving degradation when ground-truth labels are unavailable in real time?

Accepted Answer

Monitor input/prediction distribution shifts (PSI/KL) and confidence as label-free proxies

Answer

Wait for labels; nothing can be done meanwhile

Answer

Only watch CPU usage

Answer

Retrain blindly every hour

Question 29

What is the population stability index (PSI) used for in ML monitoring?

Accepted Answer

Measuring distribution shift between reference and current data to flag drift

Answer

Measuring model accuracy directly

Answer

Computing feature importance

Answer

Estimating training data size needed

Question 30

When designing a search ranking system, why might you log the propensity (probability) of each shown item being at its position?

Accepted Answer

To debias position bias via inverse-propensity weighting and enable off-policy evaluation

Answer

To compress logs

Answer

To compute latency percentiles

Answer

To detect fraud

Test yourself