Eigenvectors, SVD and PCA
Every matrix has natural axes. Eigenvectors are the directions a transformation only stretches, the SVD shows that any matrix is rotate–stretch–rotate, and PCA uses both to find the directions in which a dataset varies most.
55 min
After this lesson you can
- Find eigenvalues and eigenvectors and interpret them geometrically
- State the spectral theorem for symmetric matrices and use it
- Read the SVD as rotate–stretch–rotate and relate it to eigendecomposition
- State the Eckart–Young theorem and use truncated SVD for low-rank approximation
- Derive PCA as the top eigenvectors of the covariance, compute it via the SVD, and read explained variance
In this lesson
- Directions that don’t turn
- Symmetric matrices: the spectral theorem
- Every matrix: the singular value decomposition
- Keeping only what matters: low-rank approximation
- PCA: the natural axes of a dataset
- In code
This lesson is part of Descent Pro
Descent Pro unlocks every lesson on the route, from base camp to the summit. Every plan starts with a free trial.