Autoencoders and VAEs
Squeeze data through a narrow bottleneck and you learn a compressed code, but not a way to invent new data. The variational autoencoder fixes that by making the code a probability distribution. The ELBO derived symbol by symbol, the reparameterization trick as a picture, why VAE samples are blurry, and a real latent space you can walk through.
55 min
After this lesson you can
- Explain what an autoencoder learns, why a linear one is PCA, and why an autoencoder alone is not a generative model
- Write the VAE's generative model and inference network, and derive its ELBO both from Jensen's inequality and as log-likelihood minus a KL gap
- Derive the closed-form KL between a diagonal Gaussian and the standard normal
- Explain why sampling blocks gradients and how the reparameterization trick fixes it
- Explain why VAE samples are blurry and what the β in a β-VAE trades off
In this lesson
- Autoencoders: compression, learned
- The variational autoencoder
- The reparameterization trick
- A real latent space
- Why VAE samples are blurry
- The code
- In the papers
This lesson is part of Descent Pro
Descent Pro unlocks every lesson on the route, from base camp to the summit. Every plan starts with a free trial.