Design: Recommender System quiz

Question 1

What is the primary advantage of a two-tower architecture in large-scale recommendation systems?

Accepted Answer

It enables pre-computation of item embeddings for fast ANN retrieval

Answer

It eliminates the need for candidate generation

Answer

It provides higher accuracy than cross-encoder rankers

Answer

It solves the cold-start problem completely

Question 2

Which evaluation metric is most appropriate for a top-K recommender system where the order of recommendations matters?

Accepted Answer

NDCG@K

Answer

Precision@K

Answer

Recall@K

Answer

AUC-ROC

Question 3

What is the primary cause of popularity bias in collaborative filtering models?

Accepted Answer

Popular items appear in more training pairs, causing the model to over-recommend them

Answer

The model architecture is too simple to differentiate items

Answer

Users genuinely prefer popular items over niche ones

Answer

The loss function is not properly calibrated for large catalogs

Question 4

Which approach best addresses the cold-start problem for new items in a recommendation system?

Accepted Answer

Use content-based features to initialize new item representations

Answer

Train the model with more epochs to improve generalization

Answer

Exclude new items until they have 100 or more interactions

Answer

Apply matrix factorization with zero initialization for new items

Question 5

In the exploration-exploitation dilemma for recommendations, what does 'exploitation' refer to?

Accepted Answer

Recommending items the model is most confident the user will enjoy

Answer

Showing new items to gather more data points about the catalog

Answer

Randomly sampling from the item catalog regardless of predicted relevance

Answer

Using reinforcement learning rewards to explore user preference boundaries

Question 6

What is the best strategy for handling very long user history sequences in a transformer-based recommender?

Accepted Answer

Use linear attention to handle arbitrary-length sequences efficiently

Answer

Truncate history to the most recent N items only

Answer

Use hierarchical attention to summarize at multiple timescales

Answer

Ignore long histories as they add noise without signal

Question 7

What is the key limitation of BERT4Rec compared to SASRec for production recommendation systems?

Accepted Answer

BERT4Rec requires bidirectional attention, making autoregressive inference slow without KV caching

Answer

BERT4Rec cannot model sequential dependencies between items

Answer

BERT4Rec does not support item embeddings or categorical features

Answer

BERT4Rec cannot handle implicit feedback data

Question 8

Which negative sampling strategy is most effective for training recommenders with implicit feedback?

Accepted Answer

Hard negative mining—sampling items the model nearly ranks above the positive

Answer

Uniform random sampling from all items in the catalog

Answer

Only sampling negatives from the same category as the positive item

Answer

Treating all unobserved items as negatives with equal weight

Question 9

What does NDCG stand for and what property makes it particularly useful for evaluating recommender systems?

Accepted Answer

Normalized Discounted Cumulative Gain—it normalizes by the ideal ranking

Answer

Net Discounted Click-through Gain—it accounts for position-adjusted clicks

Answer

Normalized Deep Content Gain—it measures content diversity in rankings

Answer

Non-linear Discounted Cumulative Gradient—it measures loss gradient in rankers

Question 10

What is the main advantage of using interleaving tests over standard A/B tests for comparing recommendation algorithms?

Accepted Answer

Interleaving is more sensitive and requires far fewer users to detect differences

Answer

Interleaving tests require less engineering infrastructure to implement

Answer

Interleaving tests eliminate the need for statistical significance calculations

Answer

Interleaving tests provide better long-term retention and revenue metrics

Question 11

What is the primary role of the ranking stage in a two-stage recommendation pipeline?

Accepted Answer

To apply fine-grained scoring using rich cross-features to order a small candidate set

Answer

To retrieve the initial set of candidate items from the full billion-item catalog

Answer

To handle real-time event processing and streaming feature extraction

Answer

To manage model deployment and assign users to A/B test variants

Question 12

How should inverse propensity scoring (IPS) be applied to correct for position bias in click data?

Accepted Answer

Divide each sample's loss by the probability that the item was shown at its logged position

Answer

Multiply each click label by the item's predicted click-through rate

Answer

Add a position embedding to the ranking model's input at training time

Answer

Filter out all clicks from positions below rank 5 in the training data

Question 13

What should trigger a recommender model retraining in a production system?

Accepted Answer

Any combination of metric degradation, feature drift, catalog updates, or scheduled cadence

Answer

Only a fixed weekly schedule regardless of data or metric changes

Answer

Retraining is unnecessary if the system uses any form of online learning

Answer

Only when a new model architecture has been developed by the team

Test yourself