Bayesian inference
Stop estimating one best parameter and keep a whole distribution over it instead. Priors, posteriors and the evidence; conjugate updates you can do in your head; credible versus confidence intervals; Bayesian linear regression and its link to ridge; and why the evidence has Occam's razor built in.
60 min
After this lesson you can
- Name and compute the prior, likelihood, posterior, evidence and posterior predictive for a simple model
- Derive the Beta–Bernoulli and Gaussian–Gaussian conjugate updates, and read them as pseudo-counts and precision-weighted averages
- State precisely what a credible interval and a confidence interval each promise
- Derive the posterior of Bayesian linear regression and show that its mean is the ridge solution
- Explain how the marginal likelihood penalises flexible models (Bayesian Occam's razor), and what exact Bayesian inference costs
In this lesson
- From one number to a distribution
- Watching a belief update
- Conjugacy: the update you can do in your head
- Credible intervals and confidence intervals
- Bayesian linear regression
- The evidence, and Occam’s razor for free
- What being Bayesian costs
- In code
This lesson is part of Descent Pro
Descent Pro unlocks every lesson on the route, from base camp to the summit. Every plan starts with a free trial.