Module 7 of 10 · Beginner track
Beyond linearity — bend the model, keep the machinery
A straight line says the effect of x is the same everywhere — the tenth unit of advertising buys as much as the first. Most real relationships saturate, kink, or curve. The good news: you don't need new fitting machinery to handle that, just new features fed to the same least-squares engine. This module races the two simplest constructions:
- Step functions — chop x into bins and give each bin its own average. This is the “age bracket / income band” move every applied field makes instinctively: local, robust, readable — and blocky, with arbitrary edges.
- Polynomials — add x², x³, … and fit a single smooth formula. Elegant in the middle, but global: every data point influences the curve everywhere, and near the edges high degrees swing wildly.
Their failure modes are mirror images — one too local and jagged, the other too global and volatile — which points straight at the synthesis this chapter builds toward: splines, piecewise polynomials joined smoothly at knots. Local and smooth.
🎛 Steps vs polynomial lab
Train MSE
0.238
Test MSE
0.393
Step functions are the “dummy variable per age bracket” trick every applied field uses: chop x into bins, predict each bin's average. Local and robust, but blocky — and the bin edges are arbitrary (a tree, Module 8, learns whereto cut instead). The polynomial is smooth but global — one formula for the whole axis. Splines, this chapter's destination (covered hands-on in the advanced track), are the marriage: piecewise like steps, smooth like polynomials. Educational tool.
Where this road leads
Three extensions complete the chapter's toolkit, all covered hands-on in the advanced track. Splines fix the step/polynomial dilemma with piecewise cubics that stay smooth at the joins. Local regressiondrops formulas entirely and fits a tiny weighted line around every query point. And GAMs scale the whole idea to many inputs by giving each feature its own smooth curve and adding them up: y ≈ f₁(x₁) + f₂(x₂) + … — non-linear in every variable, yet still readable one plot per feature, which is why GAMs remain a favorite anywhere models must be explained to a committee.
Things to try
- • Give each method its best setting (sweep the slider, watch test MSE) and compare champions — on this smooth truth the polynomial usually edges the staircase, but not by much.
- • Push bins to 15: some bins hold two or three points and their steps go feral. The bin count is a bias-variance dial like every other.
- • Watch the polynomial's last half-unit near x = ±3 as you raise the degree — the edge is always where it cracks first, exactly the flaw splines were invented to fix.