Shapes, indices and sums
Read ∑ and ∏ as for-loops, count free indices to know what shape comes out, check every product's shapes, and meet einsum — index notation you can execute.
40 min
After this lesson you can
- Translate , and nested sums into loops, and back
- Distinguish free indices from summed (dummy) indices, and predict an expression's shape from them
- Check the shapes of a matrix expression — including a real one from the transformer paper — before trusting it
- Predict NumPy broadcasting, and catch the silent versus bug
- Write loops, vectorised NumPy and einsum versions of the same computation and confirm they agree
In this lesson
- A sum is a loop that adds
- Free indices and summed indices
- The shape rule
- Shape-checking a real paper
- Broadcasting: shapes that stretch
- einsum: index notation you can run
- In PyTorch
This lesson is part of Descent Pro
Descent Pro unlocks every lesson on the route, from base camp to the summit. Every plan starts with a free trial.