Start with the four outcomes
For binary classification, every prediction falls into one of four cells: true negative, false positive, false negative, or true positive. The confusion matrix keeps those counts separate, which accuracy collapses into one number.
Python
The four cells behind one accuracy number
Output
Press Run to execute this cell. The first run downloads Python; later runs reuse the session.
Bug hunt
This recall calculation uses the wrong denominator. Fix it so recall matches scikit-learn's recall_score.
Your solution
ReadyResults
Run the tests to check your solution. The first run downloads Python; later runs reuse the session.
Precision and recall answer different questions
Precision asks: of the rows predicted positive, how many were positive? Recall asks: of the actual positives, how many did we find? Improving one often costs the other when the threshold moves.
Python
Two views of the positive-class errors
Output
Press Run to execute this cell. The first run downloads Python; later runs reuse the session.
- High precision means few false alarms.
- High recall means few missed positives.
- A spam filter and a cancer screen usually need different balances.
Summarize with F1, carefully
F1 is the harmonic mean of precision and recall. It drops when either side is poor and is useful when both errors matter similarly. It is not a universal score: it ignores true negatives and assumes precision and recall deserve equal weight.
Python
Per-class precision, recall, and F1
Output
Press Run to execute this cell. The first run downloads Python; later runs reuse the session.
classification_report is a useful first look, but still report the confusion matrix when the cost of each error matters. A compact score cannot carry the full decision context.
Evaluate ranking with ROC-AUC
ROC-AUC measures how well the model ranks a random positive above a random negative across all thresholds. It is threshold-independent, so it is useful before committing to a cutoff. It does not tell you the precision at the operating point you will deploy.
Python
Ranking quality across all thresholds
Output
Press Run to execute this cell. The first run downloads Python; later runs reuse the session.
Connect the threshold to the errors
As the threshold falls, more rows become positive: recall tends to rise and precision tends to fall. Plot the tradeoff on validation data, choose an operating point that matches the cost of errors, then report the chosen threshold.
Python
The threshold is an explicit operating decision
Output
Press Run to execute this cell. The first run downloads Python; later runs reuse the session.
Choose the threshold on validation folds or a dedicated validation split. If you tune it on the test set, the final reported metric is no longer an honest estimate.
Practice
Make it stick
Run the tests as often as you like. Hints and the solution are available when you need them.
Exercise
Fit the scaled logistic regression pipeline on the training rows. Store the confusion matrix in matrix, unpack tn, fp, fn, tp, and compute precision, recall, F1, and ROC-AUC from the positive-class probabilities.
Your solution
ReadyResults
Run the tests to check your solution. The first run downloads Python; later runs reuse the session.
Quiz
Check your understanding
Lesson progress