Meaning as Coordinates

From text to numbers

So far this track has covered running a model to generate text. The next several modules cover a different, equally important use of models: turning text into numbers so a computer can measure how similar in meaning two pieces of text are - the foundation of search, retrieval, and (a couple of modules from now) retrieval-augmented generation. An embedding model does exactly that: it converts a piece of text into an embedding, a fixed-length ordered list of numbers called a vector.

You can picture each number in that list as one coordinate on a map with hundreds of directions instead of the familiar two (latitude and longitude). Text with related meaning tends to land at nearby coordinates on that map, even when it uses completely different words - "the server crashed" and "the service went down unexpectedly" might end up close together, despite sharing almost no vocabulary.

embed("the server crashed")            -> [0.12, -0.87, 0.44, ... ]  (384 numbers)
embed("the service went down")          -> [0.15, -0.81, 0.39, ... ]  (very close to the above)
embed("my favorite pizza topping")      -> [-0.9, 0.02, -0.71, ... ]  (far away on the map)

To measure exactly how close two vectors are, this track uses cosine similarity: a calculation that compares the direction two vectors point in, rather than their raw length. A higher cosine-similarity score generally means the two pieces of text are more closely related in meaning. It's important to be precise about what the vector actually gives you, though: it's useful for ranking candidate pieces of text by how related they seem, but it cannot be reversed back into the original sentence - all the specific wording is lost, only an approximate position on the "meaning map" remains.

Analogy: Imagine a library that assigns every book coordinates on a giant map of topics, based on what the book is actually about. A visitor's question gets plotted onto that same map too; the books sitting nearest to the question's coordinates become the natural first candidates to check - even if none of them happen to share the visitor's exact wording.