Normalizing flows
Take a Gaussian and bend it, one invertible step at a time, into the shape of your data. Because every step can be undone and its stretching measured, the model's density is exact. The change-of-variables formula derived in one and many dimensions, why determinants are the bottleneck, coupling layers, a real flow warping a Gaussian into two moons, and what flows can't do.
50 min
After this lesson you can
- Derive the change-of-variables formula for densities in one dimension and state it with a Jacobian determinant in many
- Explain why a flow needs invertible layers with cheap Jacobian determinants, and what a general determinant costs
- Build an affine coupling layer, write its inverse, and show its log-determinant is a sum of scales
- Train a flow by exact maximum likelihood and sample from it
- State the main limitations of flows, including why they struggle with data whose shape differs topologically from a Gaussian
In this lesson
- The change of variables, in one dimension
- Many dimensions: the Jacobian determinant
- Coupling layers
- A flow at work
- Training and sampling
- What flows can’t do
- The code
- In the papers
This lesson is part of Descent Pro
Descent Pro unlocks every lesson on the route, from base camp to the summit. Every plan starts with a free trial.