Designing a recommender system (case study) quiz

Question 1

When designing a recommender system, which strategy is most effective for addressing the cold-start problem for a brand-new user who has no interaction history?

Accepted Answer

Combine onboarding profile signals with content-based filtering and popularity fallbacks, transitioning to CF as history grows

Answer

Apply matrix factorization with zero-initialized user embeddings and train online

Answer

Serve only popularity-ranked items from the global catalog until sufficient history accumulates

Answer

Wait until the user has at least 50 interactions before making any personalized recommendations

Question 2

A media streaming service has rich item metadata (genre, cast, director) but a very sparse user-item interaction matrix. Which filtering approach is better suited and why?

Accepted Answer

Content-based filtering, because it leverages item metadata and works well with sparse interaction data

Answer

Collaborative filtering, because it captures latent tastes that go beyond item metadata

Answer

A hybrid approach is always required regardless of data density

Answer

Matrix factorization, because it handles sparse data better than either pure approach alone

Question 3

In a standard matrix factorization model for explicit ratings, what does the training objective minimize, and what regularization is typically added to prevent overfitting?

Accepted Answer

MSE between observed ratings and dot-product predictions, with L2 regularization on latent factors

Answer

Cross-entropy between binary click labels and predicted interaction probabilities

Answer

BPR pairwise loss between positive and negative item pairs, with dropout regularization

Answer

Hinge loss between user and item embeddings, with L1 sparsity regularization

Question 4

Why is Alternating Least Squares (ALS) preferred over Stochastic Gradient Descent (SGD) for large-scale implicit-feedback recommender systems deployed on distributed compute?

Accepted Answer

ALS is embarrassingly parallelizable and handles the full confidence-weighted implicit matrix via closed-form least-squares updates

Answer

ALS converges faster than SGD on explicit ratings because it uses momentum-based updates

Answer

ALS handles sparsity better by ignoring all unobserved user-item pairs entirely during each step

Answer

ALS uses mini-batches that are more cache-friendly than the sequential parameter updates in SGD

Question 5

Model A achieves higher Precision@10 but lower NDCG@10 than Model B on the same test set. What does this imply about their ranking behavior?

Accepted Answer

Model A retrieves the same relevant items as Model B but ranks them lower within the top-10 window

Answer

Model A is strictly better because Precision@10 is more interpretable for business stakeholders

Answer

NDCG@10 penalizes Model A for recommending unpopular or long-tail items

Answer

The difference is within evaluation noise; the models are equivalent in practice

Question 6

Describe the two-tower (dual encoder) architecture for candidate retrieval. What are the user tower and item tower each responsible for, and how is retrieval performed at inference time?

Accepted Answer

User and item towers independently embed features; inference uses ANN search over pre-computed item embeddings

Answer

The user tower ranks items using cross-attention over the full item catalog at inference time

Answer

Both towers share weights to ensure user and item embeddings are always in the same space

Answer

The item tower is only used at training time; at inference only the user tower produces scores

Question 7

In training a two-tower retrieval model with implicit feedback, what is the purpose of in-batch negative sampling, and what is its key limitation?

Accepted Answer

In-batch negatives reduce computation but over-represent popular items as negatives, introducing popularity bias correctable via logQ

Answer

In-batch negatives guarantee hard negatives that improve model discrimination over random sampling

Answer

In-batch negatives are only used in content-based models, not in collaborative filtering

Answer

In-batch negatives are too noisy; uniformly sampled random negatives should always be preferred

Question 8

When serving a two-tower retrieval model over a 10-million item catalog, why is exact nearest-neighbor search impractical, and which approach does FAISS use to accelerate retrieval?

Accepted Answer

FAISS uses IVF or HNSW to partition the embedding space and score only a subset of items per query

Answer

FAISS reduces dimensionality to 2D via PCA before performing exact linear search

Answer

FAISS retrains the retrieval model with fewer parameters to reduce inference compute

Answer

FAISS caches the top-1000 items per user and performs exact search only within that cache

Question 9

Your recommendation system disproportionately surfaces popular items, suppressing long-tail items relevant to niche users. Name two distinct debiasing techniques and explain the trade-off each introduces.

Accepted Answer

Inverse Propensity Scoring (high variance from rare-item weights) and popularity-penalized regularization (bias risk from misspecified propensities)

Answer

Data augmentation and early stopping to prevent the model from memorizing popular items

Answer

Removing popular items from training entirely and training only on long-tail item interactions

Answer

Increasing embedding dimension for long-tail items and decreasing it for popular items

Question 10

In an online recommendation setting, how does a contextual bandit approach balance exploration and exploitation, and why is pure greedy exploitation problematic over time?

Accepted Answer

Contextual bandits mix exploration and exploitation via UCB or Thompson Sampling, avoiding feedback loops that pure greedy selection creates

Answer

Pure exploitation is optimal because it always serves the highest-reward item given current knowledge

Answer

Contextual bandits require full user interaction history and cannot operate in real time

Answer

Exploration is only needed during the cold-start phase; once sufficient data is collected, pure exploitation is safe

Question 11

What distinguishes a session-based recommender from a traditional user-based collaborative filter, and which model architecture is best suited for session-based settings?

Accepted Answer

Session-based recommenders predict next items from within-session sequences using recurrent or self-attention architectures, without requiring persistent user profiles

Answer

Session-based recommenders use long-term user history and work best with matrix factorization

Answer

Session-based systems require user login and cannot work for anonymous users

Answer

Session-based recommenders are only applicable to e-commerce and cannot generalize to streaming or news

Question 12

Maximal Marginal Relevance (MMR) is used to increase diversity in recommendation lists. Write the MMR selection objective and explain the role of the λ parameter.

Accepted Answer

MMR maximizes relevance minus a penalty for similarity to already-selected items, with λ controlling the relevance-diversity tradeoff

Answer

MMR maximizes the sum of relevance scores across all candidate items in the window

Answer

MMR minimizes pairwise item similarity regardless of relevance to the query

Answer

MMR uses a greedy algorithm that is incompatible with embedding-based retrieval

Question 13

When A/B testing a new recommender model in production, why is user-level randomization preferred over item-level or request-level randomization?

Accepted Answer

User-level randomization maintains consistent user experience and avoids cross-contamination between treatment and control within sessions

Answer

Item-level randomization is preferred because it provides higher statistical power per experiment

Answer

Request-level randomization is best because it exposes users to both systems simultaneously, reducing sample size needs

Answer

Randomization level does not matter for recommender A/B tests because items are independent of each other

Question 14

What is the fundamental difference between explicit and implicit feedback in recommender systems, and why do most large-scale systems rely on implicit feedback?

Accepted Answer

Implicit feedback is preferred at scale because it is abundant and requires no user effort, though absence of interaction is ambiguous

Answer

Explicit feedback is always more accurate because users consciously express their true preferences

Answer

Implicit feedback is only used for cold-start users; explicit feedback is always preferred for warm users

Answer

Explicit feedback is cheaper to collect because it requires no logging infrastructure

Question 15

In a two-stage recommender (retrieval → ranking), list four distinct feature categories used by the ranking model that go beyond the user and item embeddings used in retrieval.

Accepted Answer

Cross features (user-item interaction history), contextual features (time/device), sequential features (recent history), and popularity/freshness signals

Answer

The ranking model only uses the dot-product score from the retrieval stage as its single input feature

Answer

The ranking model uses only item metadata; user features are reserved for the retrieval stage

Answer

Ranking models avoid behavioral features to prevent overfitting to historical click patterns

Question 16

Training a ranker on logged click data introduces position bias because users are more likely to click items shown at the top. Describe the IPS-based approach to correct for this bias.

Accepted Answer

Clicks are re-weighted by the inverse of the examination probability at each position, estimated via randomization experiments or a propensity model

Answer

Position bias is corrected by excluding clicks from the top 3 positions during training

Answer

Position bias is corrected by adding the item's display position as a numeric feature in the ranking model

Answer

Training on dwell time instead of clicks completely eliminates position bias without any further correction

Question 17

Bayesian Personalized Ranking (BPR) is widely used for implicit-feedback recommenders. What is the BPR objective function, and what assumption about user preference does it encode?

Accepted Answer

BPR maximizes the probability that a user prefers an observed item over an unobserved item using pairwise log-sigmoid comparison

Answer

BPR minimizes MSE between predicted scores and binary click labels for observed interactions

Answer

BPR is a listwise loss that directly optimizes NDCG using a surrogate gradient

Answer

BPR uses triplet loss with hard-negative mining to maximize the margin between positive and negative pairs

Question 18

A recommender for a video platform jointly optimizes for clicks, watch completion, and likes using multi-task learning (MTL). What is a common failure mode, and which architecture mitigates it?

Accepted Answer

A key failure mode is negative transfer where conflicting gradients degrade the primary task; MMoE mitigates this with task-specific gating over expert sub-networks

Answer

MTL always improves all tasks simultaneously because shared representations reduce overfitting

Answer

MTL requires separate fully independent models per task that are ensembled at serving time

Answer

MTL is only applicable when all tasks share the same label space and data distribution

Question 19

How do Graph Neural Networks (GNNs) improve upon standard matrix factorization for collaborative filtering, and what does LightGCN specifically remove from the standard GCN to improve performance?

Accepted Answer

LightGCN removes feature transformation matrices and non-linear activations from GCN layers, improving CF with pure neighborhood aggregation

Answer

LightGCN adds non-linear activations at each propagation layer to capture complex preference patterns

Answer

GNNs improve over MF by using cross-attention to weight item metadata features per user

Answer

LightGCN uses only first-order neighbors (direct interactions) and ignores multi-hop collaborative signals

Question 20

BERT4Rec adapts the masked language model objective for sequential recommendations. How does it train, and what is its advantage and key limitation compared to left-to-right autoregressive models like SASRec?

Accepted Answer

BERT4Rec trains by predicting randomly masked items via bidirectional attention, providing richer context than SASRec but introducing a train-test discrepancy

Answer

BERT4Rec uses causal (left-to-right) attention identical to SASRec but with larger model capacity

Answer

BERT4Rec is a GRU-based recurrent model adapted from the BERT tokenization pipeline

Answer

BERT4Rec requires explicit ratings and cannot be applied to implicit interaction sequences

Question 21

Why is a two-stage recommender pipeline (candidate generation + ranking) preferred over a single-stage model that scores all catalog items for each request?

Accepted Answer

Two-stage pipelines are used because scoring the full catalog with a complex ranker is computationally infeasible at serving latency budgets

Answer

A single-stage model is preferred because it avoids the accuracy loss introduced by candidate filtering

Answer

Candidate generation is only needed for cold-start users; warm users receive full-catalog ranking

Answer

Two-stage pipelines are required in regulated industries to ensure recommendation transparency and auditability

Question 22

A marketplace recommender systemically under-exposes items from small or new sellers compared to large established sellers. How would you detect and mitigate this disparity?

Accepted Answer

Detect disparities via disaggregated exposure metrics (e.g., Gini coefficient by seller group); mitigate via constrained optimization, quotas, or fairness-regularized training

Answer

Fairness is not a concern for recommender systems because ranking is purely meritocratic

Answer

The only fair approach is to rank all items uniformly at random to eliminate algorithmic bias

Answer

Fairness is ensured by training on a balanced dataset with equal items per seller, without modifying the ranker

Question 23

A company wants to leverage a user's e-commerce purchase history to improve recommendations on their music streaming service. What key challenges arise, and how does transfer learning address them?

Accepted Answer

Challenges include different item spaces and interaction semantics; solutions include shared user embeddings, cross-domain contrastive alignment, and meta-learning transfer

Answer

Cross-domain transfer is straightforward if both domains share the same user IDs

Answer

Cross-domain recommendations should be avoided because mixing domains always degrades single-domain performance

Answer

The simplest solution is to concatenate both interaction histories and train a single MF model across domains

Question 24

In a production recommender system, what is the distinction between batch features and real-time features, and what infrastructure is required to serve each at low latency?

Accepted Answer

Batch features are precomputed and served from a feature store (Redis/Feast); real-time features require a streaming pipeline (Kafka/Flink) with sub-second latency

Answer

Real-time features are always computed on-demand at serving time using direct database queries

Answer

Batch features are only used at training time; the serving system computes all features fresh per request

Answer

A single SQL query at request time is sufficient to compute both batch and real-time features for production recommenders

Question 25

YouTube's DNN recommender (Covington et al., 2016) describes a specific technique for handling the extreme scale of the video corpus during training. What is it and why is it necessary?

Accepted Answer

YouTube uses sampled softmax, treating recommendation as multiclass classification and sampling negatives from a noise distribution to avoid scoring all 1M+ videos

Answer

YouTube shards the video corpus across multiple models each responsible for a different genre shard

Answer

YouTube trains separate user and item towers and never combines them in a joint softmax objective

Answer

YouTube reduces scalability concerns by only recommending from the top 10,000 most popular videos

Question 26

How can a Knowledge Graph (KG) alleviate the cold-start problem for new items in a recommender system that lacks behavioral interaction data for those items?

Accepted Answer

KG-derived embeddings via TransE or RGCN provide semantically meaningful representations for new items with no interaction history, enabling cold-start retrieval

Answer

Knowledge graphs only help warm users who have existing interaction history to map onto the graph structure

Answer

Knowledge graphs replace the need for user interaction data entirely, making collaborative filtering obsolete

Answer

KG integration requires full retraining of the retrieval model whenever any new item is added to the catalog

Question 27

You deploy a LinUCB contextual bandit for real-time news recommendation. Describe the score computation and update rule, and explain the key scalability challenge when the article catalog is large.

Accepted Answer

LinUCB maintains per-arm ridge regression; scalability is challenged by separate d×d matrices per arm, addressed via shared representations or candidate pre-filtering

Answer

LinUCB scales easily to millions of arms because it uses a single global covariance matrix shared across all arms

Answer

LinUCB cannot be used for news recommendation because news items have non-stationary reward distributions

Answer

The key scalability challenge in LinUCB is the softmax over all arms, solved by approximate Bayesian inference

Question 28

In a recommender system with entity embeddings, how does feature dropout (randomly zeroing entire embedding vectors) help generalization differently from L2 regularization on embedding weights?

Accepted Answer

Feature dropout forces distributed representations by randomly masking entire embeddings, improving robustness to missing features beyond what L2 norm constraints alone provide

Answer

L2 regularization and feature dropout are equivalent because both scale embedding norms toward zero during training

Answer

Feature dropout is harmful in recommenders because item ID embeddings must always be present to produce valid predictions

Answer

L2 regularization is always preferred over dropout for embedding tables because dropout increases training instability

Question 29

The user-item interaction matrix in a typical e-commerce recommender has less than 0.1% density. What problem does this cause for neighborhood-based collaborative filtering, and how does latent factor modeling address it?

Accepted Answer

Sparse data makes user similarity estimates in neighborhood-based CF unreliable; latent factor models share statistical strength globally via dense low-rank representations

Answer

Sparsity is not a problem for neighborhood-based CF because it naturally ignores missing values in similarity computation

Answer

Latent factor models require at least 50% matrix density to produce valid embeddings

Answer

The solution to sparsity is to impute missing values with the global average rating before applying any CF method

Question 30

Your recommender is trained on click logs from the previous recommendation policy but deployed to optimize downstream purchase conversion. What covariate shift problem arises, and how does Importance Weighted ERM address it?

Accepted Answer

Covariate shift arises because training data reflects the old logging policy's exposure distribution; IW-ERM re-weights samples by new/old policy ratios to correct it

Answer

Covariate shift between click training data and purchase labels is eliminated by simply training on purchase data directly

Answer

Covariate shift is a problem only for content-based models; collaborative filtering is immune to it

Answer

The covariate shift problem is solved by increasing training data volume without modifying the loss function

Test yourself