Recommender systems at scale quiz

Question 1

Why do large-scale recommenders use a two-stage architecture (candidate generation/retrieval followed by ranking) instead of a single model scoring the whole catalog?

Accepted Answer

To split work into cheap sublinear retrieval over the full catalog and an expensive ranker over a small candidate set, meeting latency budgets

Answer

To improve interpretability of the model weights

Answer

Because a single model cannot represent both users and items

Answer

To avoid the need for embeddings entirely

Question 2

In a two-tower retrieval model, why must item embeddings be computed independently of the query (user) at serving time?

Accepted Answer

So item vectors can be precomputed and stored in an ANN index, enabling sublinear MIPS retrieval at serving time

Answer

Because user features change too slowly to matter

Answer

To reduce the embedding dimension automatically

Answer

Because dot products require both towers to share weights

Question 3

What problem does the logQ (sampled softmax) correction solve when training two-tower retrieval with in-batch negatives?

Accepted Answer

It corrects sampling bias from frequency-based in-batch negatives so the loss approximates the full-catalog softmax

Answer

It normalizes embeddings to unit length

Answer

It prevents gradient explosion in the item tower

Answer

It accounts for missing-not-at-random explicit ratings

Question 4

Why are in-batch negatives commonly used to train retrieval models, and what is their main limitation?

Accepted Answer

They provide many cheap negatives but are popularity-biased and often too easy, motivating hard-negative mining

Answer

They guarantee uniform sampling over the catalog

Answer

They eliminate the need for a ranking stage

Answer

They only work for explicit-feedback datasets

Question 5

How does the ALS (Alternating Least Squares) algorithm fit implicit-feedback matrix factorization, and why is it attractive at scale?

Accepted Answer

It alternately solves closed-form regularized least-squares for user and item factors, which parallelizes per row

Answer

It uses backpropagation through a deep network

Answer

It requires explicit 1-5 star ratings to work

Answer

It performs gradient descent on a non-convex softmax loss

Question 6

What is the core idea of Bayesian Personalized Ranking (BPR) and how does its objective differ from pointwise loss?

Accepted Answer

It maximizes a pairwise sigmoid that the positive scores higher than a sampled negative, optimizing ordering

Answer

It minimizes squared error between predicted and true rating

Answer

It is a listwise softmax over the entire catalog

Answer

It clusters users before recommending

Question 7

Why is NDCG often preferred over precision@k for evaluating recommendation ranking quality?

Accepted Answer

NDCG rewards placing relevant items higher via a positional discount and handles graded relevance

Answer

NDCG is faster to compute than precision@k

Answer

Precision@k cannot handle binary relevance

Answer

NDCG does not require ground-truth labels

Question 8

What does Recall@K measure in a retrieval/candidate-generation stage, and why is it the primary retrieval metric?

Accepted Answer

The share of relevant items captured within the top-K candidates, the key 'don't miss' metric for retrieval

Answer

The average rating error over the candidate set

Answer

The number of unique users served per second

Answer

The discounted gain of the final ranked list

Question 9

Why does HNSW give sublinear approximate nearest-neighbor search, and what is its main tradeoff?

Accepted Answer

A hierarchical small-world graph gives ~log-time greedy search at the cost of high memory and approximate recall

Answer

It exhaustively scans all vectors but with SIMD acceleration

Answer

It compresses vectors to 1 bit with no accuracy loss

Answer

It guarantees exact nearest neighbors in constant time

Question 10

How does Product Quantization (PQ) reduce memory and speed up ANN search, and what does it cost?

Accepted Answer

It encodes sub-vectors as codebook indices to shrink memory and use table lookups, at the cost of quantization error

Answer

It removes duplicate vectors from the index

Answer

It converts inner product to Euclidean distance exactly

Answer

It eliminates the need for any distance computation

Question 11

Maximum Inner Product Search (MIPS) is not directly a nearest-neighbor problem. How is it commonly reduced to one?

Accepted Answer

By transforming vectors (e.g., appending dimensions) so Euclidean NN equals MIPS, or normalizing norms

Answer

By taking the log of each coordinate

Answer

By sorting items by popularity first

Answer

MIPS cannot be approximated and must be brute-forced

Question 12

What problem does the ESMM (Entire Space Multi-task Model) solve in CVR estimation, and how?

Accepted Answer

It estimates CTR and CTCVR over all impressions and factors CVR=CTCVR/CTR to remove selection bias

Answer

It uses reinforcement learning to maximize conversions

Answer

It drops all non-converted samples to focus on positives

Answer

It is a pure retrieval model with no ranking component

Question 13

How do Multi-gate Mixture-of-Experts (MMoE) models handle conflicting objectives in multi-task ranking (e.g., clicks vs. watch-time vs. likes)?

Accepted Answer

Per-task gates softly combine shared experts, allowing specialization and reducing negative transfer

Answer

It trains one independent model per objective with no sharing

Answer

It averages all task losses with equal fixed weights

Answer

It converts all objectives into a single click label

Question 14

In the classic Wide & Deep architecture, what distinct roles do the wide and deep components play?

Accepted Answer

Wide memorizes specific cross-feature patterns; deep generalizes via embeddings to sparse/unseen combos

Answer

Wide does retrieval and deep does ranking

Answer

Both components are identical but with different learning rates

Answer

Wide handles users and deep handles items

Question 15

What does the cross network in Deep & Cross Network (DCN/DCNv2) compute that a plain MLP does not efficiently?

Accepted Answer

Explicit bounded-degree multiplicative feature crosses, parameter-efficiently, which MLPs only learn implicitly

Answer

Nearest-neighbor retrieval over item embeddings

Answer

Attention over the user's history sequence

Answer

Quantization of embedding tables

Question 16

How does DeepFM combine factorization machines with deep learning, and what is its key advantage over Wide & Deep?

Accepted Answer

FM auto-learns second-order crosses from shared embeddings, removing manual cross feature engineering

Answer

It replaces embeddings with one-hot encodings

Answer

It only models first-order linear terms

Answer

It is a sequence model for session data

Question 17

What engineering challenge dominates training DLRM-style models, and how is it typically addressed?

Accepted Answer

Massive embedding tables exceed device memory, solved via model-parallel sharding plus data-parallel MLPs and all-to-all

Answer

Vanishing gradients in deep MLPs, solved by residual connections

Answer

Slow ANN search, solved by HNSW

Answer

Overfitting, solved by dropout alone

Question 18

Why are sequential/self-attention recommenders like SASRec effective, and how does BERT4Rec differ in its training objective?

Accepted Answer

SASRec is causal next-item prediction; BERT4Rec uses bidirectional attention with a masked-item cloze objective

Answer

Both use the same left-to-right objective with different optimizers

Answer

SASRec is bidirectional and BERT4Rec is causal

Answer

Neither uses attention; both are RNN-based

Question 19

What key idea lets LightGCN outperform earlier GCN-based recommenders despite being simpler?

Accepted Answer

Dropping feature transforms and nonlinearities, keeping only linear neighbor aggregation, reduces overfitting

Answer

Adding more nonlinear layers and attention heads

Answer

Replacing the graph with a transformer

Answer

Using explicit ratings instead of implicit signals

Question 20

How does PinSage make GNN-based recommendation feasible on a multi-billion-node web-scale graph?

Accepted Answer

Random-walk neighborhood sampling with importance pooling and minibatch/MapReduce compute, avoiding full-graph ops

Answer

Loading the entire adjacency matrix into a single GPU

Answer

Replacing graph convolutions with matrix factorization

Answer

Using exact full-neighborhood aggregation for every node

Question 21

What is position bias in recommendation logs, and why does naively training a ranker on click logs reinforce it?

Accepted Answer

Higher positions get more clicks via examination, so raw-click training conflates exposure with relevance and self-reinforces

Answer

Items at the bottom always get more clicks due to recency

Answer

It only affects retrieval, never ranking

Answer

It is eliminated by adding more training data

Question 22

How does Inverse Propensity Scoring (IPS) enable unbiased off-policy evaluation of a new recommendation policy from logged data?

Accepted Answer

It reweights logged rewards by inverse logging-policy action probabilities to get an unbiased target-policy estimate

Answer

It averages rewards ignoring which policy logged them

Answer

It requires online A/B traffic to compute

Answer

It only works for explicit ratings

Question 23

Why is hard-negative mining important for retrieval models, and what risk comes with using negatives that are 'too hard'?

Accepted Answer

Hard negatives sharpen boundaries but if too hard are often false negatives (truly relevant), hurting the model

Answer

Hard negatives always improve results with no downside

Answer

Hard negatives are only used in ranking, never retrieval

Answer

Hard negatives reduce embedding dimensionality

Question 24

Why must a feature store guarantee online/offline consistency, and what failure happens if it doesn't?

Accepted Answer

To prevent train/serve skew where serving features differ from training, silently degrading live accuracy

Answer

To compress embeddings before indexing

Answer

To guarantee exact nearest neighbors

Answer

To remove the need for a ranking model

Question 25

What is the feedback (popularity) loop in deployed recommenders, and name one mitigation.

Accepted Answer

Exposure begets interactions begets more exposure, narrowing diversity; mitigate with exploration/debiasing

Answer

It is when users give too much explicit feedback; mitigate by ignoring ratings

Answer

It only occurs in cold-start and self-resolves

Answer

It is solved by increasing embedding dimension

Question 26

How do multi-armed bandit methods like Thompson sampling balance exploration and exploitation in recommendation?

Accepted Answer

It samples from each arm's reward posterior and picks the max, exploring in proportion to uncertainty

Answer

It always picks the arm with the highest observed mean

Answer

It explores uniformly at random forever

Answer

It requires the full user-item matrix to be observed

Question 27

Why does CTR-prediction calibration matter beyond ranking accuracy, and how is it commonly checked/fixed?

Accepted Answer

Downstream value/bidding uses absolute probabilities; fix miscalibration with Platt/isotonic/temperature scaling

Answer

Calibration only affects training speed, not predictions

Answer

A well-ranked model is automatically calibrated

Answer

Calibration is fixed by increasing the learning rate

Question 28

What does Maximal Marginal Relevance (MMR) optimize, and why is it used in the final re-ranking stage?

Accepted Answer

A relevance-vs-novelty tradeoff that greedily diversifies the slate to avoid redundant results

Answer

Maximum recall over the entire catalog

Answer

Minimum latency selection of candidates

Answer

Calibration of predicted probabilities

Question 29

How is the cold-start problem for new items typically handled in an embedding-based retrieval system?

Accepted Answer

Use content/side features in the item tower (plus exploration) so new items get embeddings before any interactions

Answer

Skip new items until they organically gather thousands of clicks

Answer

Assign all new items the average user embedding

Answer

Increase the ANN index size

Question 30

Why is the hashing trick used for very-high-cardinality categorical features, and what is its downside?

Accepted Answer

It bounds table size for huge vocabularies and handles new IDs, but collisions force distinct values to share embeddings

Answer

It guarantees a unique embedding per value with no collisions

Answer

It only works for numeric features

Answer

It removes the need for embeddings

Test yourself