Markov chain Monte Carlo
If you can't integrate over a posterior, sample from it. Monte Carlo averages and their 1/√n error, Markov chains that forget where they started, the Metropolis–Hastings algorithm with a proof that it works, Gibbs sampling, Hamiltonian Monte Carlo's gradient-guided leaps, and how to tell whether a chain can be trusted.
65 min
After this lesson you can
- Estimate an expectation by Monte Carlo and explain why its error falls like 1/√n in any dimension
- Define a Markov chain's stationary distribution and prove that detailed balance implies stationarity
- Implement Metropolis–Hastings, prove it satisfies detailed balance, and explain why the normalising constant cancels
- Derive Gibbs sampling as Metropolis–Hastings that always accepts
- Explain how Hamiltonian Monte Carlo uses gradients and the leapfrog integrator, and why it mixes faster on correlated targets
- Use burn-in, trace plots, autocorrelation, effective sample size and R-hat to judge convergence
In this lesson
- Monte Carlo: averages instead of integrals
- Markov chains that forget
- Metropolis–Hastings
- Gibbs sampling
- Hamiltonian Monte Carlo
- Has it converged? Diagnostics
- MCMC or variational inference?
- In code
This lesson is part of Descent Pro
Descent Pro unlocks every lesson on the route, from base camp to the summit. Every plan starts with a free trial.