Generative adversarial networks
Forget likelihood. Train a generator by making it fool a second network that is learning to catch it. The minimax game, its optimal discriminator and the Jensen–Shannon divergence hiding inside it, why the original loss had to be changed on day one, mode collapse replayed from a real training run, and the Wasserstein fix.
55 min
After this lesson you can
- State the GAN minimax objective and say what each player is trained to do
- Derive the optimal discriminator and show that the generator then minimises a Jensen–Shannon divergence
- Explain why the generator's original loss saturates and what the non-saturating loss changes
- Describe mode collapse and mode hopping, and why alternating gradient steps need not converge
- Explain the Wasserstein distance's advantage when distributions don't overlap, and how WGAN enforces the Lipschitz constraint
In this lesson
- The game
- What the discriminator learns, and what the generator then minimises
- Day-one fix: the non-saturating loss
- Watching a GAN train
- When the distributions don’t overlap: Wasserstein GAN
- The code
- In the papers
This lesson is part of Descent Pro
Descent Pro unlocks every lesson on the route, from base camp to the summit. Every plan starts with a free trial.