Support vector machines and kernels
Of all the lines that separate two classes, pick the one with the widest margin. Derive the SVM's primal and dual, see why only the support vectors matter, relax it with the hinge loss, and use kernels to draw curved boundaries — with the representer theorem explaining why this works.
50 min
After this lesson you can
- Derive the distance from a point to a hyperplane and the hard-margin SVM as a quadratic program
- Derive the SVM dual, and use complementary slackness to explain support vectors
- Write the soft-margin SVM as hinge loss plus an L2 penalty, and explain the role of
- State what makes a function a valid (positive semidefinite) kernel, and explain the RBF kernel's bandwidth
- State the representer theorem accurately, and derive kernel ridge regression from ridge regression
In this lesson
- The margin, in symbols
- The dual, and why only the support vectors matter
- Soft margins and the hinge loss
- Kernels: linear methods in a space you never build
- Why the solution lives on the data: the representer theorem
- Kernel ridge regression, derived
- In code
This lesson is part of Descent Pro
Descent Pro unlocks every lesson on the route, from base camp to the summit. Every plan starts with a free trial.