Embeddings and similarity search quiz

Question 1

What is the key difference between sparse and dense embeddings, and when would you combine them?

Accepted Answer

Sparse vectors are high-dimensional and lexical; dense are low-dimensional and semantic; hybrid fusion gets both

Answer

Sparse vectors capture semantics while dense vectors capture exact keyword matches

Answer

They are identical except for storage format

Answer

Dense embeddings are always strictly better, so combining adds no value

Question 2

Why is cosine similarity often preferred over Euclidean distance for comparing text embeddings?

Accepted Answer

Cosine ignores magnitude and matches training objectives; on normalized vectors it ranks identically to L2

Answer

Cosine is always faster to compute than Euclidean regardless of normalization

Answer

Euclidean cannot be computed on high-dimensional vectors

Answer

Cosine similarity is unaffected by the direction of the vectors

Question 3

Explain how HNSW (Hierarchical Navigable Small World) achieves fast approximate nearest neighbor search.

Accepted Answer

It uses a multi-layer navigable small-world graph with greedy descent for log-scale search

Answer

It partitions vectors into a flat grid and scans each cell exhaustively

Answer

It compresses vectors to 1 bit and brute-forces all comparisons

Answer

It clusters vectors with k-means and searches only the nearest centroid

Question 4

What is Product Quantization (PQ) and how does it reduce memory for vector search?

Accepted Answer

It splits vectors into subvectors, quantizes each with a codebook, and stores compact codes

Answer

It stores full float32 vectors but skips half the dimensions at query time

Answer

It applies PCA to reduce dimensions to exactly 2 before indexing

Answer

It hashes vectors into random buckets without any training

Question 5

What problem does the IVF (Inverted File) index solve, and what is the role of nprobe?

Accepted Answer

IVF partitions space into cells; nprobe sets how many cells to scan, trading recall for speed

Answer

nprobe sets the number of vector dimensions; IVF reduces dimensionality

Answer

IVF stores an inverted text index; nprobe is the number of query tokens

Answer

nprobe is the compression ratio used by product quantization

Question 6

Why must you use the same embedding model (and version) for indexing documents and encoding queries?

Accepted Answer

Vectors from different models are incomparable, so the whole corpus must be re-embedded on a model change

Answer

Different models produce comparable vectors as long as dimensions match

Answer

Only the query needs re-embedding when the model changes

Answer

Cosine similarity automatically normalizes across different embedding models

Question 7

What is the difference between bi-encoders and cross-encoders in semantic search?

Accepted Answer

Bi-encoders encode inputs separately for fast retrieval; cross-encoders jointly score pairs for accurate reranking

Answer

Bi-encoders are always more accurate than cross-encoders

Answer

Cross-encoders produce reusable document vectors for ANN indexes

Answer

They are two names for the same architecture

Question 8

What is the curse of dimensionality and how does it affect nearest-neighbor search?

Accepted Answer

Distances concentrate so neighbors become indistinguishable, hurting tree-based pruning

Answer

High dimensions make all distances increasingly distinct and easy to rank

Answer

It only affects sparse vectors, never dense embeddings

Answer

It speeds up search because more dimensions add more information

Question 9

Explain Matryoshka Representation Learning (MRL) and its practical benefit for retrieval.

Accepted Answer

MRL front-loads information so vector prefixes are valid lower-dimensional embeddings

Answer

MRL pads vectors with zeros to standardize their length

Answer

MRL encrypts embeddings layer by layer for privacy

Answer

MRL requires retraining whenever you change the truncation length

Question 10

In FAISS, what is the difference between IndexFlatIP and IndexFlatL2, and how do you do cosine similarity?

Accepted Answer

Normalize vectors and use IndexFlatIP, since dot product of unit vectors equals cosine

Answer

Use IndexFlatL2 directly; it computes cosine by default

Answer

Cosine is impossible in FAISS; you must use an external library

Answer

IndexFlatIP automatically normalizes vectors before scoring

Question 11

What are the trade-offs of recall vs. latency in approximate nearest neighbor (ANN) systems?

Accepted Answer

Higher exploration (efSearch/nprobe) raises recall but increases latency; tune to the SLA

Answer

ANN indexes always return exact results with no recall loss

Answer

Recall and latency are independent and cannot be traded off

Answer

Latency only depends on vector dimensionality, not search parameters

Question 12

What is the role of a reranker in a RAG retrieval pipeline, and why not just retrieve more candidates?

Accepted Answer

Rerankers add precision to top candidates; retrieving more alone adds noise and token cost

Answer

Rerankers reduce recall, so retrieving more candidates is always better

Answer

Rerankers replace the embedding model entirely

Answer

Rerankers only reorder by document length

Question 13

How does chunking strategy affect embedding-based retrieval quality?

Accepted Answer

Large chunks dilute meaning while tiny chunks lose context; overlap and semantic boundaries help

Answer

Chunk size is irrelevant because embeddings normalize length

Answer

Smaller chunks are always strictly better for retrieval

Answer

Overlap between chunks always degrades retrieval quality

Question 14

What does it mean for an embedding space to be anisotropic, and why is it a problem?

Accepted Answer

Embeddings cluster in a narrow cone, inflating similarities; whitening or contrastive training helps

Answer

Anisotropy means embeddings are perfectly uniform and well-separated

Answer

It is a property of sparse vectors only

Answer

It improves discrimination by spreading vectors apart

Question 15

Compare dot product, cosine, and Euclidean distance as similarity metrics for embeddings.

Accepted Answer

Dot product uses magnitude+angle, cosine only angle; on normalized vectors L2 ranks like cosine

Answer

All three always give identical rankings regardless of normalization

Answer

Cosine is the only metric affected by vector magnitude

Answer

Euclidean distance ignores magnitude by definition

Question 16

What is Maximum Inner Product Search (MIPS) and why can't it directly reuse metric-tree structures?

Accepted Answer

Inner product is not a metric, breaking triangle-inequality-based trees; use transforms or graph indexes

Answer

MIPS is just Euclidean search, so metric trees apply directly

Answer

MIPS minimizes inner product and is identical to L2 search

Answer

MIPS only works on sparse vectors

Question 17

How would you evaluate the quality of an embedding model for a retrieval task?

Accepted Answer

Use ranking metrics (recall@k, MRR, nDCG) on labeled data plus benchmarks like MTEB/BEIR

Answer

Only training loss is needed to judge embedding quality

Answer

Accuracy on a classification head is the standard retrieval metric

Answer

Perplexity is the correct metric for embeddings

Question 18

What is contrastive learning and why is it central to training modern embedding models?

Accepted Answer

It pulls positives together and pushes negatives apart so similarity reflects semantics

Answer

It trains a classifier to predict the next token

Answer

It is a clustering algorithm applied after embedding

Answer

It maximizes reconstruction error of an autoencoder

Question 19

Why are hard negatives important when fine-tuning embedding models, and how are they typically mined?

Accepted Answer

They are close-but-wrong items that sharpen boundaries, mined via top retrieved non-relevant docs

Answer

Hard negatives are random documents added for regularization

Answer

Hard negatives are duplicate positives used to balance batches

Answer

They are only useful for cross-encoders, not bi-encoders

Question 20

What is binary quantization of embeddings and what accuracy-recovery technique is commonly paired with it?

Accepted Answer

It maps dims to bits (~32x smaller, Hamming distance) and rescoring with float vectors recovers accuracy

Answer

Binary quantization keeps full precision and needs no rescoring

Answer

It doubles vector size to improve recall

Answer

It replaces cosine similarity with exact L2 on bits

Question 21

How do late-interaction models like ColBERT differ from standard single-vector dense retrieval?

Accepted Answer

ColBERT keeps per-token vectors and scores with MaxSim, trading storage for token-level accuracy

Answer

ColBERT uses one vector per document like standard dense retrieval

Answer

ColBERT is a sparse lexical method with no embeddings

Answer

ColBERT requires joint encoding at query time like a cross-encoder

Question 22

What is the 'lost in the middle' problem and how does retrieval ordering relate to it?

Accepted Answer

LLMs underweight middle-of-context info, so put top chunks at the start/end and rerank

Answer

LLMs attend uniformly across context, so ordering is irrelevant

Answer

The problem is solved by simply adding more chunks

Answer

It only affects the embedding model, not the LLM

Question 23

Why might you apply dimensionality reduction (e.g., PCA) to embeddings before indexing, and what is the risk?

Accepted Answer

It cuts storage and speeds search but risks dropping discriminative directions and hurting recall

Answer

PCA increases recall guaranteed because it removes only noise

Answer

Dimensionality reduction has no effect on search latency

Answer

PCA must always be fit on the query set, not the corpus

Question 24

What is the difference between symmetric and asymmetric semantic search, and how does it affect model choice?

Accepted Answer

Symmetric matches similar-form texts; asymmetric matches short queries to long docs, needing task-matched models

Answer

Symmetric and asymmetric search require identical models in all cases

Answer

Asymmetric search means using two different distance metrics

Answer

Symmetry only refers to whether the matrix of similarities is symmetric

Question 25

How do instruction-tuned / prefix-based embedding models (e.g., E5, Instructor) improve retrieval?

Accepted Answer

Task prefixes/instructions tailor embeddings to intent; using the trained prefixes is essential

Answer

Prefixes are decorative and have no effect on the output vector

Answer

They replace the need for any similarity metric

Answer

Instructions are only used at index time, never for queries

Question 26

What is Reciprocal Rank Fusion (RRF) and why is it popular for hybrid search?

Accepted Answer

RRF fuses ranked lists using 1/(k+rank), needing only ranks, not comparable scores

Answer

RRF averages raw similarity scores from each retriever

Answer

RRF requires both retrievers to output identical score scales

Answer

RRF only works with two lists and cannot scale to more

Question 27

What are the main challenges of keeping a vector index fresh under frequent inserts, updates, and deletes?

Accepted Answer

HNSW deletes are costly (tombstones, rebuilds); segments and versioned reindexing manage freshness

Answer

Vector indexes update in place with zero overhead for deletes

Answer

Inserts are impossible in ANN indexes; only full rebuilds work

Answer

Deletes are free but inserts always require a full rebuild

Question 28

How can metadata filtering interact badly with ANN search (the 'filtered search' problem)?

Accepted Answer

Pre/post-filtering can break navigability or drop valid neighbors; filter-aware indexes help

Answer

Filtering never affects ANN recall because it is applied independently

Answer

Filters always improve recall by reducing the search space

Answer

Post-filtering is guaranteed to return exactly k results

Question 29

Why are embeddings typically L2-normalized before being stored in a vector database?

Accepted Answer

It makes dot product equal cosine and standardizes score ranges by removing magnitude

Answer

Normalization changes the semantic meaning of the vector

Answer

It is required only for sparse vectors

Answer

Normalization increases vector dimensionality

Question 30

What is recall@k as a metric for ANN systems, and how do you obtain ground truth for it?

Accepted Answer

Recall@k is the fraction of true top-k neighbors returned, with ground truth from exact brute-force

Answer

Recall@k measures human relevance judged by annotators

Answer

Recall@k requires no reference and is read from the index directly

Answer

Recall@k measures query latency in milliseconds

Test yourself