Module 1 of 10 · Beginner track
Learning from data
Every dataset you will ever meet is a mix of two ingredients: a pattern(how advertising spend relates to sales, how income relates to default risk, how yesterday's volatility relates to today's) and noise(everything else — measurement error, luck, the thousand factors you didn't record). Statistical learning is the craft of recovering the pattern without being fooled by the noise.
People come to it wanting two different things, and it's worth knowing which one you want. Prediction: “give me the best guess for a new case — I don't care how.” Inference: “tell me whichinputs matter and by how much — the understanding is the product.” A black-box model can be superb at the first and useless for the second; much of this track (especially the regression modules) is about tools that do both.
Before any formulas, calibrate your eyes. Below are three mystery datasets, all equally noisy. One hides a straight line, one hides a curve, and one hides nothing whatsoever. Fit both models to each and watch the two R² numbers.
🎛 Three-dataset mystery lab
R² on training data
0.02
R² on fresh data
-0.03
Dataset A hides a straight line, B hides a curve, C hides nothing at all— pure noise. Notice that the flexible fit reports a decent training R² on dataset C anyway: give a flexible model 40 random points and it will find a “pattern.” Only the fresh-data column tells the truth. Educational tool.
The one lesson to carry out of Module 1
On dataset C — pure noise — the flexible model still reports a respectable training R². It found a “pattern” because a flexible enough model always finds one; that is what flexibility means. The only number that told the truth was performance on fresh data the model never saw. Every module that follows is, one way or another, an elaboration of this single fact — and it is exactly why a spectacular trading backtest, fitted and judged on the same history, proves nothing yet.
Things to try
- • On dataset B, compare the two fits: the straight line misses the curve (both R² mediocre but honest); the flexible fit captures it (both R² high). Flexibility used well.
- • On dataset C, note the gap: training R² positive, fresh-data R² near zero or negative. Flexibility used badly.
- • Before toggling on each dataset, guess from the scatter alone which one has real signal. Harder than it should be — that's the point.