What generative modelling is
A classifier learns to answer questions about data; a generative model learns the data itself, as a probability distribution you can sample from. Why that is so much harder, the five tricks that make it possible, what each family of models can and cannot do, and how anyone decides whether a generator is good.
45 min
After this lesson you can
- Distinguish learning from learning , and sampling from density evaluation
- Explain why normalisation makes density modelling hard in high dimensions, and name the trick each model family uses to cope
- Place autoregressive models, flows, VAEs, diffusion, energy-based models and GANs on the explicit/implicit map, with their trade-offs
- Convert a negative log-likelihood to bits per dimension, and explain why good likelihood does not guarantee good samples
- State the FID formula, what it measures, and at least three of its caveats
In this lesson
- What “learning a distribution” asks for
- Five ways around the normaliser
- How do you know a generator is any good?
- The code
- A map of this part
- In the papers
This lesson is part of Descent Pro
Descent Pro unlocks every lesson on the route, from base camp to the summit. Every plan starts with a free trial.