Embeddings, or how meaning becomes geometry
Turn a word into a list of numbers, and distance becomes meaning.
Give a word coordinates, and similarity stops being a feeling; it becomes a distance you can compute.
Computers do not read; they count. So before a model can do anything useful with language, the language has to become numbers. An embedding is that translation: it takes a word, a sentence, or an image and maps it to a vector, a long list of numbers, that marks a single point in a high-dimensional space. The whole art is in arranging that space so position means something.
Why a list of numbers
Picture a flat map where every word is a pin. "Coffee" and "tea" sit near each other; "galaxy" sits far away in another quarter. A real embedding does this not in two dimensions but in hundreds or thousands, far more room to separate the many shades of meaning a word can hold. Each dimension is a faint, nameless feature the model found useful; together they fix a location precise enough that closeness genuinely tracks similarity.
Crucially, no one writes these numbers by hand. The model learns them by reading enormous amounts of text and noticing which words keep company with which: the old linguist's intuition that you shall know a word by the company it keeps. Words used in similar contexts drift toward similar coordinates, so the geometry falls out of usage, not from a dictionary.
Meaning stops being a definition you look up and becomes a location you can measure.
Distance is similarity, direction is analogy
Once meaning lives in space, two operations do almost all the work. Distance answers "what is most like this?" and that single question is the engine under semantic search, recommendation, and clustering. It is also the retrieval step in RAG: your question is embedded, the store is searched for the nearest passages, and those are handed to the model as context. The match is on meaning, which is why a query and a document that share no words can still be neighbours.
Direction carries something subtler. Because the model learns regular structure, certain relationships become consistent moves through the space: the step from "man" to "woman" runs roughly parallel to the step from "king" to "queen." So "king" − "man" + "woman" lands in the neighbourhood of "queen." It is not magic and not exact, but a clean illustration of a messy truth: relationships are encoded as directions you can add and subtract. Get the geometry right, and reasoning about meaning becomes arithmetic.