Precision, recall and ROC
Why 99% accuracy can mean a useless model. The confusion matrix, precision and recall, the F-score, thresholds, ROC and precision–recall curves (and what the area under each really measures), averaging over many classes, and the everyday metrics for regression.
50 min
After this lesson you can
- Build a confusion matrix and read accuracy, precision, recall, specificity and the false-positive rate off it
- Explain why accuracy misleads when classes are imbalanced, and why precision depends on prevalence while recall does not
- Derive the F-score as a weighted harmonic mean and write it in terms of counts
- Trace ROC and precision–recall curves by sweeping a threshold, and prove that ROC AUC is the probability that a random positive outranks a random negative
- Choose between macro, micro and weighted averages for many classes, and between MSE, MAE and R² for regression
In this lesson
- Four kinds of outcome
- Two questions, two denominators
- One number from two: the F-score
- Scores, thresholds and the trade-off
- The ROC curve
- Precision–recall curves
- Many classes: macro, micro, weighted
- Regression metrics
- In code
This lesson is part of Descent Pro
Descent Pro unlocks every lesson on the route, from base camp to the summit. Every plan starts with a free trial.