Matrix calculus you actually need
The chain rule with the bookkeeping done by vectors and matrices. That means one layout convention, the Jacobian, five identities you'll reuse forever, and the habit of checking every derivative by its shape and by finite differences.
50 min
After this lesson you can
- State the layout convention used in Descent and recognise the others in papers
- Compute Jacobians and read them as the best local linear map
- Derive the gradients of , , and a linear layer
- Apply the chain rule as a product of Jacobians, and as a vector–Jacobian product
- Verify any gradient numerically with finite differences
In this lesson
- First, a convention (and a warning)
- The Jacobian: the derivative of a vector function
- Five identities, derived
- The chain rule, with Jacobians
- Check shapes, then check numbers
This lesson is part of Descent Pro
Descent Pro unlocks every lesson on the route, from base camp to the summit. Every plan starts with a free trial.