Embeddings Explained: How Meaning Becomes a List of Numbers
What embeddings are, how they place similar things near each other in vector space, and how they power semantic search, recommendations and modern AI.
Computers can't compare meanings, but they're excellent at comparing numbers. Embeddings bridge that gap: each word, sentence, image, product or user is assigned a point in a high-dimensional space — a vector of, say, 384 or 768 numbers — *learned* so that similar things land near each other. 'Doctor' ends up close to 'nurse' and far from 'volcano'. Once meaning is geometry, finding related things reduces to finding nearby points.
Where the coordinates come from
Nobody hand-assigns them. Embeddings fall out of training a model on a task that requires understanding similarity. Classic word embeddings were trained to predict a word from its neighbours — words used in similar contexts got pulled toward each other. Modern sentence and image embeddings come from neural networks (often transformers) trained on huge corpora, sometimes with *contrastive* objectives that explicitly pull matching pairs (a photo and its caption) together while pushing mismatched pairs apart. The dimensions aren't individually meaningful — the geometry as a whole is what encodes the structure.
Measuring 'near': cosine similarity
The standard similarity measure is cosine similarity — the cosine of the angle between two vectors: 1 for pointing the same way, 0 for unrelated, negative for opposed. It ignores vector length and compares direction only, which is usually what you want for meaning. The famous arithmetic result — *king − man + woman ≈ queen* — showed that early word-embedding spaces even encoded relationships as consistent directions.
What embeddings power
- Semantic search — embed the query and all documents; return the nearest documents. Finds 'how to fix a flat tyre' when you searched 'puncture repair', with zero shared keywords.
- Recommendations — users and items in one space; recommend the items nearest to a user's vector.
- Retrieval-augmented generation (RAG) — chatbots fetch the most relevant passages by embedding similarity before answering.
- Clustering and deduplication — near-identical items sit almost on top of each other.
- Anomaly detection — a point far from every cluster is worth a look.
At scale: vector databases
Comparing a query against a billion vectors one by one is too slow, so production systems use approximate nearest-neighbour (ANN) indexes — trading a sliver of accuracy for orders-of-magnitude speed. That's the core of what a 'vector database' does.
Embeddings are also the front door of every transformer — the first layer converts tokens to vectors before attention goes to work. Drill the details on the embeddings and similarity search topic page, then test yourself in **AI Learning** — illustrated guides, cheat sheets and ~10,000 practice questions, fully offline with no account.
Free · Works offline · From word vectors to RAG — learn the ideas behind modern AI.
Sources
Frequently asked questions
More in Learn AI & Data Science
- Coding Interview Strategy: A Framework for Solving Problems Under PressureKnowing algorithms isn't enough — you have to deploy them under pressure. Here's a repeatable framework for solving unseen problems and a study plan that builds durable skill.
- P vs NP and Complexity Classes: What 'Hard' Really Means (Intuition, No Proofs)Why do some problems have fast algorithms and others resist every attempt? P vs NP is the deepest open question in computer science — here's the intuition, minus the proofs.
- Union-Find (Disjoint Set Union): Near-Constant Connectivity with Two OptimizationsUnion-Find answers 'are these two things connected?' and 'connect them' in almost O(1) — thanks to two beautifully simple optimizations that make the trees nearly flat.