Retrieval and reranking quiz

Question 1

What is the primary advantage of BM25 over dense vector retrieval for keyword-heavy queries?

Accepted Answer

BM25 handles exact keyword and rare-term matches better than embedding-based retrieval

Answer

BM25 is faster at inference due to GPU-accelerated index lookups

Answer

BM25 supports multilingual retrieval without any language-specific configuration

Answer

BM25 produces dense vector representations compatible with HNSW indexes

Question 2

Why are cross-encoders used for reranking rather than first-stage retrieval over large corpora?

Accepted Answer

Cross-encoders score each query-document pair jointly, which is too slow for first-stage retrieval over large corpora

Answer

Cross-encoders cannot be fine-tuned on domain-specific data

Answer

Cross-encoders require pre-computing all document embeddings at index time

Answer

Cross-encoders only work with BM25 features, not raw text inputs

Question 3

What property of HNSW (Hierarchical Navigable Small World) makes it the preferred ANN index for most production retrieval systems?

Accepted Answer

HNSW employs a multi-layer graph for logarithmic query time with high recall and supports online insertions

Answer

HNSW uses product quantization to compress vectors to 4 bytes each, enabling billion-scale in RAM

Answer

HNSW guarantees exact nearest-neighbor results with zero recall loss

Answer

HNSW requires periodic full re-indexing because it does not support dynamic insertions

Question 4

Given results from a BM25 search and a dense vector search, how does Reciprocal Rank Fusion (RRF) combine them, and why is it preferred over linear score combination?

Accepted Answer

RRF assigns scores based on reciprocal rank positions across lists, summing 1/(k+rank) per list, avoiding scale incompatibility

Answer

RRF multiplies the BM25 score by the cosine similarity score for each document

Answer

RRF picks the union of both lists and re-ranks using a cross-encoder

Answer

RRF requires both retrievers to return exactly the same number of results

Question 5

What is the 'MaxSim' operation in ColBERT and why does it enable more precise matching than single-vector bi-encoders?

Accepted Answer

MaxSim computes, for each query token, the maximum similarity over all document tokens, then sums these maxima

Answer

MaxSim averages all pairwise query-document token cosine similarities

Answer

MaxSim applies BM25 term frequencies weighted by contextual embeddings from BERT

Answer

MaxSim is used only during the reranking stage and not during first-stage retrieval

Question 6

How does SPLADE produce learned sparse representations, and why can it outperform BM25 on out-of-vocabulary queries?

Accepted Answer

SPLADE learns sparse vocabulary-space representations via BERT + log(1+ReLU) and FLOPS regularization, enabling implicit query expansion while remaining inverted-index compatible

Answer

SPLADE passes TF-IDF weights through a neural network to densify sparse representations

Answer

SPLADE produces dense 768-dim embeddings compatible with HNSW via vocabulary projection

Answer

SPLADE requires a separate cross-encoder reranking step because its sparse scores are uncalibrated

Question 7

What problem does Hypothetical Document Embeddings (HyDE) solve, and what is its primary risk?

Accepted Answer

HyDE uses an LLM to generate a hypothetical answer, embeds it, and retrieves via document-space proximity—with hallucination propagation as the main risk

Answer

HyDE reduces retrieval latency by caching hypothetical embeddings for frequent query templates

Answer

HyDE replaces the reranker by using cross-attention between hypothetical and real documents

Answer

HyDE applies only to multilingual retrieval where query and document languages differ

Question 8

What does Matryoshka Representation Learning (MRL) enable, and how is it achieved during training?

Accepted Answer

MRL trains one model whose first d dimensions form a valid d-dim embedding at each nested granularity, by summing losses at multiple dimensionalities during training

Answer

MRL trains separate model checkpoints for each embedding size to minimize quality loss per dimension

Answer

MRL adds a progressive bottleneck that forces the model to compress all information into the first 64 dimensions

Answer

MRL applies quantization-aware training to reduce embedding precision from float32 to int8

Question 9

What does the BEIR benchmark measure, and what key finding did it reveal about early dense retrieval systems?

Accepted Answer

BEIR is a heterogeneous zero-shot benchmark across 18 diverse datasets, revealing that early dense retrievers underperform BM25 out-of-domain

Answer

BEIR only evaluates retrieval on MS MARCO passage and document tracks

Answer

BEIR is an in-domain benchmark measuring retrieval quality after domain-specific fine-tuning

Answer

BEIR measures latency and throughput of ANN search engines such as FAISS and ScaNN

Question 10

How is NDCG@10 computed, and what role does the logarithmic discount factor play?

Accepted Answer

NDCG@10 divides the discounted cumulative gain (DCG) by the ideal DCG, applying a log rank discount that penalizes lower-ranked relevant documents

Answer

NDCG@10 is the fraction of relevant documents found in the top 10 results

Answer

NDCG@10 only works with binary relevance labels (relevant / not relevant)

Answer

NDCG@10 measures recall at rank 10, normalized by total relevant documents in the corpus

Question 11

Why are hard negatives critical for training dense retrieval models, and what is 'false negative contamination'?

Accepted Answer

Hard negatives are topically similar but non-relevant documents that provide strong gradient signal; false negatives are mislabeled relevant documents that corrupt training

Answer

Hard negatives are sampled uniformly at random from the corpus to increase batch diversity

Answer

Hard negatives are the same documents as positives but with labels flipped to test model robustness

Answer

Hard negatives are only used in sparse retrieval models like BM25 and SPLADE

Question 12

How does product quantization (PQ) reduce memory for billion-scale vector indexes, and what accuracy trade-off does it introduce?

Accepted Answer

PQ splits each vector into subspaces and quantizes each independently to a centroid code, approximating distances with precomputed lookup tables at reduced memory

Answer

PQ compresses vectors by applying SVD and storing only the top principal components

Answer

PQ reduces dimension by averaging adjacent embedding dimensions, trading recall for speed

Answer

PQ is lossless compression that reduces memory without any recall degradation

Question 13

What problem does Maximal Marginal Relevance (MMR) address in document retrieval, and how is the iterative selection carried out?

Accepted Answer

MMR iteratively selects a diverse result set by maximizing a weighted relevance-minus-redundancy objective

Answer

MMR is a loss function for training rerankers with listwise pairwise comparisons

Answer

MMR computes mean relevance rank across multiple query reformulations

Answer

MMR is a dimensionality reduction technique applied to embeddings before ANN indexing

Question 14

How does RAG-Fusion improve upon naive single-query RAG, and what role does RRF play?

Accepted Answer

RAG-Fusion generates multiple query variants, retrieves for each, then merges results with RRF for robustness to ambiguity

Answer

RAG-Fusion fine-tunes the retrieval model on query-answer pairs to improve dense retrieval

Answer

RAG-Fusion uses a fusion layer to combine query and document embeddings before generation

Answer

RAG-Fusion is a training method that combines sparse and dense retrieval losses in one objective

Question 15

Compare pointwise, pairwise, and listwise LLM reranking strategies. Which is most token-efficient and why?

Accepted Answer

Listwise reranking is most token-efficient because a single LLM call re-orders all k documents simultaneously, amortizing prompt overhead across candidates

Answer

Pointwise reranking is most token-efficient because it uses the shortest prompt per document

Answer

Pairwise reranking is most token-efficient because it needs only n/2 comparisons

Answer

All three strategies use the same total tokens when applied to the same document set

Question 16

What does Anthropic's 'Contextual Retrieval' technique add to standard RAG chunking, and what metric improvement was reported?

Accepted Answer

Contextual Retrieval prepends LLM-generated context summaries to each chunk before indexing, reducing retrieval failures by ~49% (contextual embeddings + BM25) and ~67% with reranking added

Answer

Contextual Retrieval removes stopwords from chunks to improve BM25 precision

Answer

Contextual Retrieval uses larger 4096-token chunks instead of the standard 512-token size

Answer

Contextual Retrieval applies cross-encoder scoring at index time to pre-rank all chunk pairs

Question 17

What is 'late chunking' in the context of long-context embedding models, and how does it differ from standard pre-embedding chunking?

Accepted Answer

Late chunking embeds the full document with a long-context model and derives chunk embeddings from pooled token-level representations, preserving cross-chunk context

Answer

Late chunking delays chunking until query time, splitting based on the query's topic

Answer

Late chunking is another name for the parent document retriever pattern in LangChain

Answer

Late chunking applies BM25 scoring after dense retrieval to filter irrelevant chunks

Question 18

What problem does ColPali solve that traditional text-based RAG pipelines struggle with, and how does it work?

Accepted Answer

ColPali embeds document page images using a vision-language model and retrieves via MaxSim over patch embeddings, bypassing OCR entirely

Answer

ColPali extracts text from PDFs using neural OCR and then applies ColBERT for token-level retrieval

Answer

ColPali is a cross-encoder that jointly encodes query text and document images for reranking

Answer

ColPali compresses PDF pages into 512-dim embeddings via a CNN, compatible with standard HNSW

Question 19

What is the InfoNCE contrastive loss used to fine-tune embedding models, and why does in-batch negative scaling matter?

Accepted Answer

InfoNCE maximizes positive-pair similarity relative to in-batch negatives; larger batches provide harder training signal and better representations

Answer

InfoNCE maximizes cosine similarity between all pairs in a batch equally

Answer

InfoNCE is a ranking loss using binary cross-entropy between query and positive document

Answer

InfoNCE applies L2 regularization to embedding weights to prevent representation collapse

Question 20

How does an IVF (Inverted File Index) approximate nearest-neighbor search, and what is the nprobe parameter?

Accepted Answer

IVF partitions vectors into k-means clusters at index time; at query time, only nprobe clusters are searched to approximate nearest neighbors

Answer

IVF compresses each vector using product quantization and stores only the centroid IDs at index time

Answer

IVF performs exact nearest-neighbor search in parallel across all vectors using GPU acceleration

Answer

IVF applies locality-sensitive hashing (LSH) to project vectors into hash buckets for sub-linear search

Question 21

Compare pseudo-relevance feedback (PRF) query expansion with LLM-based query expansion. When would you prefer each?

Accepted Answer

PRF extracts terms from top retrieved docs (fast, no LLM); LLM expansion generates related terms and sub-questions (better for ambiguous queries, at higher latency cost)

Answer

PRF uses a neural model to predict query intent from user click-through data

Answer

LLM-based query expansion is always better than PRF because it uses a larger model with world knowledge

Answer

Both PRF and LLM expansion require fine-tuning the retrieval model on the expanded queries

Question 22

What is the 'parent document retriever' pattern in RAG, and what dual problem does it solve?

Accepted Answer

The parent document retriever indexes small chunks for precise retrieval but returns larger parent documents as LLM context

Answer

The parent document retriever embeds full documents and returns only the most relevant individual sentences

Answer

The parent document retriever stores large chunks for retrieval but returns small chunks as LLM context

Answer

The parent document retriever uses a hierarchical index where parent nodes are k-means cluster centroids

Question 23

What does a self-query retriever accomplish that standard semantic similarity retrieval cannot, and what LLM task is involved?

Accepted Answer

Self-query retrieval uses an LLM to decompose queries into a semantic search string and structured metadata filters enforced on the vector store

Answer

Self-query retrieval embeds the query multiple times with different random seeds for ensemble retrieval

Answer

Self-query retrieval applies the query against itself to eliminate redundant sub-questions

Answer

Self-query retrieval performs closed-book QA without any document retrieval step

Question 24

What does MTEB measure and why is it the authoritative leaderboard for embedding model selection in 2025–2026?

Accepted Answer

MTEB evaluates embedding models across 8 task types and 50+ datasets in 100+ languages, serving as the authoritative embedding model selection benchmark

Answer

MTEB only evaluates retrieval tasks on English text, similar to MS MARCO

Answer

MTEB is a latency benchmark measuring inference speed of embedding models on standard hardware

Answer

MTEB is a fine-tuning dataset for training multilingual sentence transformers

Question 25

Beyond RRF, what is 'relative score fusion' (RSF) for hybrid retrieval, and when does it outperform RRF?

Accepted Answer

RSF normalizes scores to [0,1] and computes a weighted sum, preserving score magnitude—outperforming RRF when scores are well-calibrated

Answer

RSF is identical to RRF but uses a different constant k value (k=100 instead of 60)

Answer

RSF re-encodes documents using both sparse and dense models simultaneously in a joint forward pass

Answer

RSF applies a cross-encoder to re-score all documents after RRF merging for a three-stage pipeline

Question 26

What is the architecture of BGE-Reranker and how does it differ from a BGE bi-encoder retriever?

Accepted Answer

BGE-Reranker encodes query and document jointly through a BERT encoder, producing a scalar relevance score—requiring one forward pass per candidate

Answer

BGE-Reranker is a bi-encoder that uses a larger model dimension (1536-dim) than the standard BGE retriever

Answer

BGE-Reranker applies product quantization to BGE embeddings for faster similarity computation

Answer

BGE-Reranker is a generative model that produces relevance explanations alongside scalar scores

Question 27

What training signal does the original DPR model use, and what was its key finding relative to BM25?

Accepted Answer

DPR trains dual BERT encoders with in-batch and BM25-hard negatives, outperforming BM25 by 9–19 pp on open-domain QA top-20 retrieval

Answer

DPR uses a single shared encoder for both queries and passages to reduce model parameters

Answer

DPR trains with cross-entropy loss using only random negatives uniformly sampled from Wikipedia

Answer

DPR uses token-level MaxSim interaction similar to ColBERT for higher retrieval accuracy

Question 28

Explain the sentence window retrieval strategy and its typical benefit over naive fixed-size chunk retrieval.

Accepted Answer

Sentence window retrieval indexes sentences for precise retrieval but expands to surrounding sentences as LLM context, improving faithfulness vs. fixed-size chunking

Answer

Sentence window retrieval splits documents into overlapping 512-token chunks with a 50-token stride for overlap

Answer

Sentence window retrieval is a reranking technique that scores sentences individually before score aggregation

Answer

Sentence window retrieval uses a sliding window of cross-encoder scores to find the most relevant passage

Question 29

What does Mean Reciprocal Rank (MRR) measure, and what is its key limitation compared to NDCG?

Accepted Answer

MRR is the mean of reciprocal ranks of the first relevant document per query, ignoring all subsequent relevant documents

Answer

MRR measures the fraction of queries for which at least one relevant document appears in the top 10

Answer

MRR computes the average cosine similarity between query embeddings and top-1 document embeddings

Answer

MRR normalizes DCG by the ideal ranking and averages across all queries

Question 30

When should you use a hosted reranking API (e.g., Cohere Rerank, Jina Reranker) versus a self-hosted cross-encoder?

Accepted Answer

Hosted APIs suit ease-of-use and no GPU infra scenarios; self-hosted is better for data privacy, low latency, domain fine-tuning, or high-volume cost optimization

Answer

Hosted reranking APIs always outperform self-hosted models because they use larger proprietary models

Answer

Hosted reranking APIs are never suitable for production because they add unacceptable network latency

Answer

Self-hosted cross-encoders cannot be fine-tuned on domain data due to licensing restrictions

Test yourself