Module 17 · Neighbors & structure
Graphical models — who directly talks to whom
Correlation matrices lie by transitivity. If A drives B and B drives C, then A and C are correlated — even though A never touches C. In a portfolio, in a sensor network, in a set of economic indicators, the correlation matrix shows you a dense tangle where the true wiring might be a sparse chain. The question worth asking is not “do A and C move together?” but “do they still move together once everything else is held fixed?”
For Gaussian data that sharper question has a beautiful, exact answer sitting in the precision matrix — the inverse of the covariance. Its off-diagonal entries give partial correlations, and a zero entry means the two variables are conditionally independent: whatever association you saw between them was entirely routed through the others. Draw a node per variable and an edge per nonzero precision entry, and you have the dependency graph — the model is the graph.
The lab wires up five variables in a ring and hands you the direct links. Watch both matrices at once: local changes in the graph produce global changes in correlation, but the partial correlations stay local and honest.
🎛 Dependency-graph lab
Estimated graph (edges where |partial corr| > 0.08)
Correlation vs partial correlation, per pair
| Pair | Correlation | Partial corr. | Direct edge? |
|---|---|---|---|
| A–B | 0.71 | 0.45 | ✓ |
| A–C | 0.58 | 0.00 | — |
| A–D | 0.58 | 0.00 | — |
| A–E | 0.71 | 0.45 | ✓ |
| B–C | 0.71 | 0.45 | ✓ |
| B–D | 0.58 | 0.00 | — |
| B–E | 0.58 | 0.00 | — |
| C–D | 0.71 | 0.45 | ✓ |
| C–E | 0.58 | 0.00 | — |
| D–E | 0.71 | 0.45 | ✓ |
Marginal correlationanswers “do these two move together?” — and A&C correlate even with no chord, because influence flows through B. Partial correlationasks the sharper question: “do they still move together once every other variable is held fixed?” It comes straight off the inverse covariance (precision) matrix, and a zero there means conditionally independent— no edge. Slide the A–C chord up from zero and watch the direct edge appear; kill B–C and watch A–C's correlation shrink while its partial stays put. Estimating which precision entries are zero from data (the graphical lasso) is how these graphs are learned in practice. Educational tool.
Learning the graph from data
With real data you don't know the precision matrix — you estimate it, and a raw inverse of a sample covariance is dense (every entry slightly nonzero) and, when features outnumber observations, doesn't exist at all. The graphical lasso is the standard fix: maximize the Gaussian likelihood with an L1 penalty on the precision entries — the exact selection trick from Module 3, now zeroing edgesinstead of coefficients. The penalty dial sweeps from a fully-connected hairball to an empty graph; somewhere in between is the sparse skeleton worth interpreting. A useful equivalent view: each variable's neighbors are the variables that a (lasso) regression of it on all the others keeps.
Things to try
- • With the A–C chord at 0, read the A–C row of the table: solid correlation, near-zero partial, no edge. The correlation is pure through-traffic via B.
- • Raise the chord to 0.3: the A–C edge snaps into the graph and its partial correlation comes alive — a genuinely new direct channel.
- • Cut B–C to 0 and watch A–C's correlation drop too — the indirect route died — while unrelated pairs barely move.
- • Sweep the threshold slider: this is the graphical lasso's penalty in miniature — too low and noise edges appear, too high and real ones vanish.