Module 4 of 10 · Beginner track
Classification — the model proposes, the threshold decides
When the outcome is a category — default / no default, spam / not spam — regression's first cousin takes over. Logistic regression keeps the linear recipe but pushes it through a squash: P(yes | x) = 1 / (1 + e^−(β₀+β₁x₁+…)), so the output is a legitimate probability between 0 and 1. Each coefficient now moves the log-odds: a one-unit change in x₁ multiplies the odds of “yes” by e^β₁.
Here's the underappreciated part: the model's job ends at the probability. Turning 0.31 into a yes-or-no actionrequires a threshold, and the threshold is a business decision, not a statistical one, because the two mistakes it trades off almost never cost the same. Missing a genuine default (false negative) and hassling a good customer (false positive) have different price tags — so “accuracy” alone is nearly always the wrong scoreboard.
🎛 Threshold & ROC lab
True positive
99
False positive
65
False negative
21
True negative
215
Sensitivity (recall)
83%
of real positives, caught
Specificity
77%
of real negatives, cleared
Precision
60%
of flags, correct
Predicted probabilities (negatives ↑, positives ↓)
ROC curve — every threshold at once
The model produces the probabilities; you choose where to cut them. Slide the threshold and watch the errors trade: left catches more positives but flags more clean cases; right is the opposite. The red dot rides the ROC curve — the menu of every achievable (false-positive, true-positive) pair; a better model bows the curve toward the top-left. At low prevalence, note precision sag even when sensitivity looks fine — the base-rate effect that makes rare-event screening hard. Educational tool.
The vocabulary, mapped to the lab
- Confusion matrix — the four tiles: the two ways to be right (TP, TN) and the two ways to be wrong (FP, FN).
- Sensitivity vs specificity — of the real positives, how many did we catch? Of the real negatives, how many did we clear? The threshold slides one up as it slides the other down.
- Precision and the base rate — of everything we flagged, how much was real? At 5% prevalence even a good model's flags are mostly false alarms; screening for rare events is hard arithmetically, before it's hard statistically.
- ROC curve — every threshold's (false-positive rate, true-positive rate) pair on one curve; the red dot is where your current threshold sits. The area under it (AUC) summarizes the model independent of any threshold — 0.5 is coin-flipping, 1.0 is perfection.
Things to try
- • Play the medical screener: prevalence 5%, threshold low enough for ~95% sensitivity — now look at precision. Most flags are false. That's not a bad model; it's a low base rate.
- • Improve the model (separation up): the histograms pull apart, the ROC bows toward the corner, and every threshold gets a better menu. Model quality and threshold choice are separate dials.
- • Find the threshold where FP and FN counts are equal — then argue why your application would not want that one. (It almost never does.)