Module 3 of 10 · Beginner track
Linear regression — beat the machine if you can
Linear regression proposes the simplest useful story: y ≈ β₀ + β₁x — a baseline level plus a constant effect per unit of x. Fitting it means choosing the two numbers, and the criterion is refreshingly concrete: make the residuals— each point's vertical miss from the line — collectively as small as possible, measured by the residual sum of squares (RSS). Before reading another word about it, try to minimize RSS yourself.
🎛 Fit-the-line lab
Your RSS
125.0
sum of squared red segments
Best possible RSS
21.7
the least-squares line
Your R² / best R²
-0.42 / 0.75
variance explained
The red segments are the residuals— each point's vertical miss from your line — and RSS is the sum of their squares. Least squares is nothing more than the (β₀, β₁) making that sum as small as it can be; squaring is why one far-off point tugs harder than several near-misses. Get your RSS within 5% of optimal and the tile turns green. R² re-expresses the same fit as the fraction of the y-variance the line explains. Educational tool.
What the machine knows that your thumbs don't
The least-squares solution isn't found by search — calculus hands it over in closed form (the optimal slope is the covariance of x and y over the variance of x). Two properties worth internalizing from your slider session: squaring punishes big misses disproportionately, so the line bends toward outliers more than feels fair; and around the optimum the RSS surface is flat — many nearby lines score almost identically, which is precisely why the fitted coefficients come with standard errors rather than certainty.
With several inputs the same machinery becomes y ≈ β₀ + β₁x₁ + … + βₚxₚ, and each coefficient's meaning sharpens to: the effect of nudging this input with all the others held fixed. That “held fixed” clause is where the classic traps live — correlated inputs fighting over credit, confounders making spurious effects — and it's why coefficient signs can flip when a new variable enters. The advanced track's regression modulepicks up exactly there, with shrinkage as the modern fix.
Things to try
- • Chase the green tile: get within 5% of optimal RSS by hand, then reveal the answer and see how close your (β₀, β₁) landed.
- • Deliberately set a too-steep slope and watch which residual segments grow — the endpoints complain first. That leverage is why extreme-x points dominate a fit.
- • Note how R² restates your RSS: it's just the fraction of y's variance the line accounts for — 0 for a flat line at the mean, 1 for a perfect fit.