Maybe you’ve heard the term embeddings floating around. Maybe you know it has something to do with documents, RAG, or “contextual similarity.” But to most, it sounds like one of those fancy gatekeeping words that separates engineers from the business team.
It’s actually much simpler than it sounds.
From Words to Math
Last week, we discussed how a chatbot splits your prompt into tokens (the “internal format” of language). But there is one final step before the AI can actually “think”: each token gets assigned a specific numerical value.
Think of it like a giant, multidimensional dictionary. Your prompt is broken into parts, and those parts are looked up in the model’s master list to find their corresponding numbers.
- Your Prompt: “Word”
- Tokenization:
["wo", "rd"] - Embedding:
[324, 123](The numerical “ID” for those pieces)
In the math world, this list of numbers is called a vector. This is the only language the model actually speaks. Every operation the AI performs—predicting the next word, summarizing a deck, or translating a memo—is just the model adding, subtracting, and multiplying these lists of numbers.
The Magic of “Neighborhoods”
Because these numbers aren’t assigned randomly, they have a superpower: Location.
In the world of embeddings, words with similar meanings end up with similar numbers. If “Apple” is represented by [1.1, 2.2], then “Pear” might be [1.2, 2.1]. They are numerically “close” to each other.
This makes it trivial for a computer to compare two documents. It doesn’t look for matching letters; it looks for matching math. If the numbers are close, the meaning is close.
Why BD should care
This is the engine behind the “Contextual Matching” revolution. It’s why AI search is so much better than the old-school keyword search. It’s also why I am incredibly optimistic about the future of advertising.
Instead of showing an ad because a user typed the word “shoes,” we can now show an ad because the mathematical intentof their entire conversation suggests they are planning a hiking trip.
Leave a Reply