Gaussian processes
Put the prior directly on functions. A Gaussian process says how similar a function's values at any two inputs should be; conditioning on data is exact Gaussian algebra, the marginal likelihood tunes the kernel, and infinitely wide neural networks turn out to be Gaussian processes too.
60 min
After this lesson you can
- Define a Gaussian process by its mean and covariance functions, and read a kernel as a prior belief about functions
- Derive the GP regression posterior mean and covariance from Gaussian conditioning
- Show that Bayesian linear regression is a GP with a particular kernel
- Choose kernel hyperparameters by maximising the marginal likelihood, and explain its fit-versus-complexity trade-off
- State the O(n³) cost, name the main approximations, and connect GPs to infinitely wide networks
In this lesson
- From weights to functions
- Definition, and kernels as beliefs
- Conditioning on data
- Learning the kernel from the marginal likelihood
- The cost, and how to pay less
- Infinitely wide networks are Gaussian processes
- In code
This lesson is part of Descent Pro
Descent Pro unlocks every lesson on the route, from base camp to the summit. Every plan starts with a free trial.