Embeddings, retrieval and RAG
A language model knows only what was in its training data. Retrieval-augmented generation looks up relevant text at question time and puts it in the prompt. That needs text turned into vectors, similarity turned into search, and search made fast enough for millions of documents — and it fails in specific, well-documented ways.
55 min
After this lesson you can
- Explain what a text embedding is and why cosine similarity turns retrieval into nearest-neighbour search
- Decode the contrastive loss used to train dense retrievers, including in-batch negatives
- Compute BM25 by hand and say when lexical retrieval beats dense retrieval
- Describe how IVF, product quantization and HNSW make nearest-neighbour search fast, and what they give up
- Assemble a RAG pipeline (chunk, embed, index, retrieve, rerank, generate) and name its documented failure modes
In this lesson
- Text as points in space
- Where good embeddings come from
- The old way still works: BM25
- Finding neighbours fast
- Retrieval-augmented generation
- How RAG fails
This lesson is part of Descent Pro
Descent Pro unlocks every lesson on the route, from base camp to the summit. Every plan starts with a free trial.