Designing a recommender system (case study)

The most effective strategy combines content-based recommendations from onboarding profile signals (stated preferences, demographics) with popularity-based fallbacks (trending or editorially curated items). As the user accumulates interactions, the system gradually transitions to collaborative filtering. Some systems also use a bandit approach to efficiently explore user preferences during the early session.

Content-based filtering is better suited because it relies on item features rather than requiring a dense user-item matrix, and can make personalized recommendations from just a few user interactions by matching item attributes to user preference profiles. Collaborative filtering would suffer from the sparsity problem, producing unreliable similarity estimates due to insufficient co-rated items between users.

The objective minimizes mean squared error (MSE) between observed ratings and the dot product of user and item latent factor vectors (R ≈ U·Vᵀ). L2 regularization (λ‖U‖² + λ‖V‖²) is added on the latent factors to prevent overfitting to the sparse observed ratings. The regularization strength λ is a key hyperparameter tuned via cross-validation on held-out ratings.

def mf_loss(users, items, ratings, U, V, lam=1e-4):
    preds = (U[users] * V[items]).sum(dim=1)
    mse = ((preds - ratings) ** 2).mean()
    reg = lam * (U[users].norm(dim=1).pow(2).mean() + V[items].norm(dim=1).pow(2).mean())
    return mse + reg

ALS alternates between fixing user factors and solving for item factors as closed-form least-squares problems, which are highly parallelizable across users and items. For implicit feedback (Hu, Koren & Volinsky 2008), the full confidence-weighted matrix must be incorporated for each user/item update, and ALS exploits the linear algebra structure to do this efficiently. SGD on implicit feedback requires careful negative sampling and lacks the same embarrassingly parallel structure at Spark/Hadoop scale.

Model A places relevant items in the top-10 window but tends to rank them toward the bottom of that window (e.g., positions 8–10), while Model B surfaces relevant items near the top (positions 1–3). NDCG applies a logarithmic discount to lower-ranked positions, making it sensitive to the ordering of relevant items within the window, not just their presence. For user experience, Model B is likely superior since users rarely scroll past the first few results.

The user tower encodes user features (ID, history, context) into a dense embedding vector, while the item tower independently encodes item features (ID, metadata) into a matching embedding space. Both towers are trained jointly so that dot-product or cosine similarity between user and item embeddings predicts interaction probability. At inference, all item embeddings are pre-computed and indexed in an ANN store (e.g., FAISS), and a single user embedding query retrieves the top-K candidates in milliseconds.

class TwoTowerModel(nn.Module):
    def __init__(self, user_dim, item_dim, emb_dim):
        super().__init__()
        self.user_tower = nn.Sequential(nn.Linear(user_dim, 256), nn.ReLU(), nn.Linear(256, emb_dim))
        self.item_tower = nn.Sequential(nn.Linear(item_dim, 256), nn.ReLU(), nn.Linear(256, emb_dim))

    def forward(self, user_feat, item_feat):
        u = F.normalize(self.user_tower(user_feat), dim=-1)
        v = F.normalize(self.item_tower(item_feat), dim=-1)
        return (u * v).sum(dim=-1)

In-batch negative sampling uses the positive items from other training examples within the same mini-batch as negatives for each user-positive pair, requiring no additional forward passes since all negatives are already computed in the batch. The key limitation is sampling bias: popular items appear more frequently as positives across training examples and are therefore over-represented as negatives, causing the model to under-penalize popular items and introducing popularity bias. A logQ correction (subtracting log item sampling frequency from logits) is commonly applied to counteract this.

Exact nearest-neighbor search requires computing a dot product for every item per query — O(N·d) work per query, which at 10M items and typical embedding dimensions (128–256) means billions of multiply-accumulates per request, far exceeding real-time latency budgets (< 50 ms). FAISS primarily uses inverted file index (IVF) — partitioning the embedding space into Voronoi cells so only a subset of items is scored — or HNSW (hierarchical navigable small-world graphs), both achieving sub-linear query time with a controllable recall/latency trade-off.

Inverse Propensity Scoring (IPS) up-weights long-tail items in the loss by the inverse of their exposure probability — but it inflates variance because rare items have very high weights and few training examples. Regularization-based approaches (e.g., SNIPS normalized weighting, or a popularity-penalizing loss term) reduce variance but may introduce bias if propensity estimates are misspecified. Both require careful calibration: overly aggressive debiasing reduces short-head accuracy to gain modest long-tail gains, hurting aggregate CTR.

A contextual bandit models expected reward for each item given the current user context and uses policies like UCB, Thompson Sampling, or epsilon-greedy to occasionally recommend high-uncertainty items (exploration) while mostly serving high-expected-reward items (exploitation). Pure exploitation collapses into a feedback loop: items that appear to perform well receive more exposure, making it impossible to discover better items or maintain accurate reward estimates for underexplored options. Over time, this degrades recommendation quality and entrenches popularity bias.

Session-based recommenders predict the next item based solely on the interaction sequence within the current (often anonymous) session, without relying on a persistent user profile or long-term history. Traditional collaborative filtering requires sufficient user history to build a meaningful embedding. Recurrent models (GRU4Rec) and self-attention models (SASRec, BERT4Rec) are well-suited because they capture sequential dependencies and positional patterns within a session.

At each step, MMR selects the item maximizing λ·sim(item, query) − (1−λ)·max_{j∈Selected} sim(item, j). Here λ∈[0,1] controls the trade-off: λ=1 gives pure relevance ranking with no diversity, while λ=0 maximizes diversity regardless of relevance. Values of λ=0.5–0.7 are common in production, tuned via A/B testing on long-session engagement metrics.

def mmr(remaining, selected, query_emb, embs, lam=0.6):
    scores = []
    for i in remaining:
        rel = cosine_sim(embs[i], query_emb)
        div = max((cosine_sim(embs[i], embs[j]) for j in selected), default=0.0)
        scores.append((lam * rel - (1 - lam) * div, i))
    return max(scores)[1]

User-level randomization ensures each user is consistently exposed to either the control or the treatment system, preventing interference effects where a single user encounters both systems in the same session. Item-level or request-level randomization creates inconsistent within-session experiences, inflates variance, and makes causal attribution of behavioral changes unreliable. User-level splits better satisfy the Stable Unit Treatment Value Assumption (SUTVA) required for valid causal inference.

Explicit feedback consists of direct user-provided ratings or reviews, which are sparse because most users never rate items they consume. Implicit feedback includes behavioral signals like clicks, watch time, purchases, and dwell time — these are abundant and naturally collected at scale without user effort. The challenge with implicit feedback is that the absence of interaction is ambiguous (not interested vs. not exposed), requiring careful modeling of confidence and exposure.

The ranking model typically incorporates: (1) cross features — user-item interaction features such as historical CTR on the item's category; (2) contextual features — time of day, device type, location; (3) sequential features — the user's recent interaction sequence (last N items viewed or purchased); and (4) popularity and freshness features — real-time CTR, recency, trending score. These enable the ranker to capture nuanced relevance signals that the efficiency-optimized retrieval stage cannot express.

In Inverse Propensity Scoring (IPS), each click is re-weighted by 1/P(examined | position k), where the examination probability at each rank position is estimated from randomization experiments (e.g., result swapping) or a position propensity model. This inverse-propensity weighting removes position-induced bias from the training signal so the ranker learns true item relevance rather than display-position artifacts. The Unbiased Learning-to-Rank (ULTR) framework extends this by jointly estimating propensities and relevance via an EM-style alternating optimization.

BPR maximizes the probability that a user prefers an observed (positive) item over an unobserved (negative) item, formalized as maximizing ∑ ln σ(x̂_ui − x̂_uj) over positive-negative pairs (minus an L2 regularization term). It encodes the assumption that the relative ordering between a consumed and a non-consumed item is informative about preference, even without knowing the absolute satisfaction level. This pairwise objective is more suitable for ranking tasks than pointwise MSE, which treats missing interactions as zero-rating observations.

def bpr_loss(pos_scores, neg_scores):
    # pos_scores, neg_scores: (batch,) tensors of predicted scores
    return -torch.log(torch.sigmoid(pos_scores - neg_scores)).mean()

MTL trains shared lower layers (shared user and item encoders) with separate task-specific heads, allowing the model to learn complementary representations. A common failure mode is negative transfer: gradients from different tasks pull shared parameters in opposing directions, degrading performance on the most important task (e.g., watch completion) to benefit a noisier signal (e.g., clicks). The Multi-gate Mixture of Experts (MMoE) architecture mitigates this by learning task-specific gating networks over a pool of expert sub-networks rather than forcing a single shared representation.

GNNs propagate embeddings through the user-item interaction graph so each node's representation incorporates multi-hop neighborhood information, capturing higher-order collaborative signals that plain MF cannot. LightGCN removes the feature transformation weight matrices (W) and non-linear activation functions from standard GCN layers, finding they add noise without improving CF performance. The final embedding is a weighted sum (typically uniform) of node representations across all propagation layers.

def lightgcn_layer(emb, adj_norm):
    # adj_norm: D^{-1/2} A D^{-1/2} sparse normalized adjacency
    # No weight matrix, no activation — pure neighborhood aggregation
    return torch.sparse.mm(adj_norm, emb)

BERT4Rec randomly masks items in the user interaction sequence and trains the model to predict the masked items using bidirectional self-attention over the full sequence context. This bidirectional context allows richer representation learning than left-to-right models like SASRec, which can only attend to past items. The key limitation is a train-test discrepancy: at inference, BERT4Rec masks only the final position (the target to predict), while training masks arbitrary positions — this mismatch must be managed carefully to avoid performance degradation.

Scoring all catalog items (e.g., 10M+) with a complex ranking model at each request is computationally infeasible within real-time latency budgets (typically < 100 ms end-to-end). Candidate generation uses a lightweight model (two-tower ANN retrieval) to narrow the catalog to O(100–1000) candidates, and the ranking stage applies a more expressive model (deep neural net with cross-features) only to those candidates. This decouples retrieval efficiency from ranking quality, allowing each stage to be independently optimized.

Detection involves computing exposure metrics disaggregated by seller group — e.g., measuring the Gini coefficient of item impressions or comparing average rank position by seller-size quantile. Mitigation strategies include constrained optimization (adding exposure fairness constraints to the ranking objective), post-processing re-ranking (minimum exposure quotas per seller segment), or fairness-regularized training loss that penalizes disparate impact. Care must be taken to distinguish fairness constraints from legitimate quality filtering to avoid unnecessarily harming user experience.

Key challenges include different item spaces (products vs. songs), different interaction semantics (purchase vs. listen/skip), and potentially limited cross-domain user overlap. Transfer learning approaches include: (1) shared user embedding training — learning a universal user representation across domains with multi-task objectives; (2) cross-domain contrastive alignment — mapping domain-specific embeddings to a shared space; and (3) meta-learning initialization — bootstrapping the target-domain model from the source domain. Cross-domain signals are most valuable for cold-start users in the target domain who have rich history in the source domain.

Batch features (e.g., long-term user preferences, historical item statistics) are pre-computed offline and stored in a low-latency feature store (Redis, Cassandra, or Feast) for O(1) lookup at serving time. Real-time features (e.g., items clicked in the last 10 seconds, current session context) require a streaming pipeline (Kafka + Flink/Spark Streaming) to compute and push updates with sub-second latency. The feature store must support both modes — offline batch writes and online real-time updates — with a unified API to avoid train-serve skew.

YouTube uses sampled softmax during training, treating recommendation as a multiclass classification problem (predicting which video the user watches next) and sampling a small subset of the 1M+ video corpus as negative classes rather than computing the full softmax over all videos. This is analogous to noise contrastive estimation (NCE) and makes the training objective computationally tractable without sacrificing model quality significantly. The item serving frequency is used as the noise distribution, allowing a logQ correction to recover an approximately unbiased gradient estimate.

A KG links items to structured attributes and entities (e.g., a new movie → director → genre → related movies) via relational triples. Item embeddings can be initialized from KG structure using methods like TransE or RGCN, providing a non-zero, semantically meaningful representation even for items with zero interaction history. At inference, the KG-based embedding can be queried against user embeddings using the same ANN index, enabling immediate non-trivial recommendations before behavioral data accumulates.

LinUCB maintains per-arm ridge regression parameters Aₐ = I + XₐᵀXₐ and bₐ = Xₐᵀrₐ, with θₐ = Aₐ⁻¹bₐ. The UCB score is θₐᵀxₜ + α√(xₜᵀAₐ⁻¹xₜ); after observing reward rₜ, Aₐ and bₐ are updated incrementally. The scalability challenge is maintaining a separate d×d matrix per arm: with 100K articles and d=100 features, storing all Aₐ matrices requires ~4 GB and inverting each one is O(d³) per arm, making online scoring over the full catalog prohibitive. Practical solutions include disjoint LinUCB with a shared representation, neural bandits, or restricting to a pre-filtered candidate set from the retrieval stage.

L2 regularization constrains the magnitude of embedding vectors, preventing them from growing arbitrarily large but still allowing the model to memorize rare feature co-occurrences encoded in the direction of the embedding. Feature dropout randomly zeroes out entire embedding vectors during training, forcing the model not to rely on any single feature and encouraging distributed representations that generalize to missing or unseen features at inference. This is particularly valuable for rare user-item interactions where ID embeddings could otherwise be memorized.

With 0.1% density, most user pairs share very few or zero co-rated items, making cosine or Pearson similarity estimates statistically unreliable and producing poor neighbor sets. Latent factor models (e.g., matrix factorization) compress the sparse matrix into low-rank representations (U ∈ ℝ^{m×k}, V ∈ ℝ^{n×k}) where dense latent dimensions capture global co-occurrence patterns, sharing statistical strength across all users and items. This generalization enables meaningful recommendations even for users with minimal interaction history.

Covariate shift occurs because training data consists of items exposed and clicked under the old logging policy, while the new model must score items including those rarely shown before — the item distribution at training time differs from what the new policy will encounter. Importance Weighted ERM (IW-ERM) re-weights each training sample by P_new(item) / P_old(item) to correct for the shift between the logging and target policies, which is essentially off-policy correction from the counterfactual evaluation literature. Propensity clipping is applied to control variance from extreme importance weights.

A weighted sum collapses objectives into a single scalar (w₁·CTR + w₂·revenue + w₃·retention) and finds one optimal solution per weight vector, requiring practitioners to pre-specify trade-off weights before optimization. Pareto optimization finds the entire frontier of non-dominated solutions — where no objective can be improved without degrading another — allowing business stakeholders to choose the operating point post-hoc. Pareto approaches are preferred when the relative importance of objectives is uncertain or changes over time, such as during seasonal campaigns. Note that weighted sum can only recover solutions on the convex portion of the Pareto frontier.

Business rules during re-ranking include: diversity constraints (no more than K items from the same category), freshness boosts (promote items published in the last 24 hours), inventory constraints (demote out-of-stock items), sponsored item insertion (interleave paid slots at fixed positions), and safety filters (suppress flagged content). These are enforced via rule-based post-processing on a priority queue (heap) and slot-allocation algorithms, or formulated as a constrained optimization problem for complex interdependent constraints.

BM25 retrieves based on exact term-frequency overlap (sparse, lexical matching) and generalizes poorly to semantic paraphrases or vocabulary mismatch. Dense retrieval uses a bi-encoder to embed queries and items into a shared dense space, enabling semantic similarity matching. The key training challenge is constructing high-quality hard negatives: random negatives are too easy and the model trivially separates them, while mining hard negatives (items close to positives in embedding space but not relevant) requires iterative ANN search during training, which is computationally expensive.

The three most critical metrics are: (1) Catalog coverage and diversity — detects model collapse to a narrow set of popular items, which can be invisible in aggregate click metrics; (2) CTR and conversion rate disaggregated by user segment — detects performance regressions or distribution shift affecting specific cohorts (e.g., new users, mobile users); (3) Recommendation serving latency at P99 — detects infrastructure degradation (e.g., ANN index overload, feature store timeouts) that causes quality to silently degrade via fallback logic. Anomaly detection alerts on all three catch failures not visible in aggregate engagement numbers.