Design: Search Ranking quiz

Question 1

What is the primary difference between BM25 and dense retrieval in search ranking?

Accepted Answer

BM25 uses term-frequency lexical matching; dense retrieval uses neural embeddings for semantic similarity

Answer

BM25 is a neural model; dense retrieval is rule-based

Answer

BM25 requires GPU acceleration; dense retrieval runs on CPU

Answer

BM25 handles semantic queries better than dense retrieval

Question 2

How does a cross-encoder re-ranker differ from a bi-encoder retriever in neural search?

Accepted Answer

A cross-encoder scores query-document pairs jointly with full attention, enabling richer relevance but slower inference

Answer

A bi-encoder is more accurate than a cross-encoder for ranking

Answer

A cross-encoder can be used for large-scale retrieval because it pre-computes embeddings

Answer

A bi-encoder is only applicable to image search, not text search

Question 3

How is position bias measured and corrected in search click data used for training?

Accepted Answer

Documents shown at higher ranks receive more clicks regardless of true relevance, biasing click-based training

Answer

Position bias only affects visual search and not text search systems

Answer

Position bias is not a concern in modern neural search systems

Answer

Position bias means the search engine penalizes documents from unknown websites

Question 4

How should freshness be incorporated into search ranking?

Accepted Answer

Classify queries as time-sensitive or evergreen and apply freshness weighting only where recency matters

Answer

Apply maximum freshness weighting to all queries regardless of type

Answer

Freshness should never be used as a ranking signal because it conflicts with relevance

Answer

Index documents once per week to ensure consistency across all query types

Question 5

What is Reciprocal Rank Fusion and when should it be used in hybrid search?

Accepted Answer

RRF is a parameter-free score fusion method that combines ranked lists by reciprocal rank position

Answer

RRF requires training a neural network on relevance labels

Answer

RRF is only applicable to image retrieval systems, not text search

Answer

RRF always outperforms learned fusion methods regardless of data availability

Question 6

What is ColBERT and how does it improve over standard bi-encoder retrieval?

Accepted Answer

ColBERT encodes documents at token level and uses MaxSim late interaction for fine-grained relevance

Answer

ColBERT uses a single document embedding like standard bi-encoders but with a larger model

Answer

ColBERT is a cross-encoder that requires joint query-document attention for every candidate

Answer

ColBERT was designed specifically for image search rather than text retrieval

Question 7

What is the role of anchor text in web search ranking?

Accepted Answer

Anchor text provides editorial descriptions of linked pages, supplementing content signals in ranking

Answer

Anchor text is used only for navigation menus and has no effect on ranking

Answer

Anchor text is always spam and should be excluded from ranking features

Answer

Anchor text is only useful for image search indexing

Question 8

How do you monitor a production search ranking system for quality degradation?

Accepted Answer

Monitor serving health, engagement metrics, automated offline evaluation, and coverage metrics continuously

Answer

Only monitor server CPU and memory usage for search quality

Answer

Quality monitoring only requires running A/B tests when engineers suspect a problem

Answer

Monitor only the zero-result rate as the single key search quality indicator

Question 9

How does contextual retrieval improve search quality for ambiguous or short queries?

Accepted Answer

Contextual retrieval uses session history, user profile, or LLM-generated hypothetical documents to improve ambiguous query resolution

Answer

Contextual retrieval only applies to conversational search systems, not standard web search

Answer

Contextual retrieval is less accurate than standard keyword search for short queries

Answer

Contextual retrieval requires storing every user's full browsing history in plaintext

Test yourself