Embeddings
What is it?
Embeddings turn text, images, or other data into lists of numbers that capture meaning, so similar things end up close together.
Explain like I'm 5
Why was it created?
Computers can't compare meaning directly. Embeddings were created to represent meaning as numbers so machines can measure how similar two things are.
Where is it used?
- Semantic search
- Recommendations
- Retrieval-augmented generation (RAG)
- Clustering and similarity
Why should developers care?
Embeddings power search, recommendations, and the retrieval step in RAG, so they're a key tool for building AI features.
How does it work?
A model converts an input into a vector — a list of numbers — positioned so that similar inputs have nearby vectors. You can then compare items by measuring the distance between their vectors.
Real-world example
A search feature embeds your query and all documents, then returns the documents whose vectors are closest in meaning — even if they use different words.
Common use cases
- Meaning-based search
- Finding similar items
- Powering RAG retrieval
- Grouping related content
Advantages
- Capture meaning, not just keywords
- Enable similarity comparisons
- Work across text, images, and more
- Foundation for semantic search and RAG
Disadvantages
- Quality depends on the model
- Vectors are hard for humans to interpret
- Need a vector store to search at scale
- Can encode biases from training data
When should you use it?
When you need to compare or search by meaning rather than exact words.
When should you avoid it?
For exact-match lookups where simple keyword search or a database index is enough.
Alternatives
Related terms
Interview questions
Beginner
- What is an embedding?
- Why represent text as numbers?
Intermediate
- How do you measure similarity between embeddings?
- How are embeddings used in search?
Senior
- How do embeddings enable RAG?
- What are the risks of bias in embeddings?
Common misconceptions
- "Embeddings store the original text" — they store a numeric representation of meaning, not the words themselves.
- "Closer vectors are always more relevant" — similarity is approximate and depends on the embedding model.
Fun facts
- In embedding space, simple math can capture analogies like 'king − man + woman ≈ queen'.
- Two passages about the same idea can have close embeddings despite sharing no words.
Timeline
- 2013 — Word embeddings popularized by word2vec
Learning resources
Quick summary
Embeddings represent data as numeric vectors capturing meaning, so similar items sit close together — enabling semantic search and RAG.
Cheat sheet
- Data turned into meaning-vectors
- Similar items are close together
- Compared by vector distance
- Power search, recommendations, RAG