Module 5
Performance: measure, then vectorize
Rule zero: measure before you optimize. Every senior engineer has a story about a week spent tuning a function that turned out to be 2% of the runtime. The profile is always surprising — the hot spot is a JSON parse, a logging call, an accidental O(n²) — almost never the loop you suspected. Three tools cover the whole job:
# micro: time ONE expression, best-of-N, interference-resistant python -m timeit -s "xs = list(range(10_000))" "sum(xs)" # macro: profile a whole run, then browse the flame graph python -m cProfile -o out.prof my_script.py snakeviz out.prof # production: sample a LIVE process — no restart, no code change py-spy top --pid 12345
timeit for micro-benchmarks, cProfile + snakeviz for “where does my program spend its time?”, and py-spy when the slow thing is already running in production. Only once the profile has named the culprit do you reach for the two big levers below.
Lever one: vectorize.A pure-Python loop over numbers is slow for a structural reason, not a “Python is bad” reason: every + is a dynamic dispatch through object headers on heap-boxed floats. A numpy operation is one C loop over a contiguous typed array — the per-element interpreter tax vanishes, and on numerics that is typically worth 50–200×. Here is the same rolling mean both ways:
# pure Python: O(n·w) iterations, every one interpreted
means = []
for i in range(w - 1, len(prices)):
s = 0.0
for j in range(i - w + 1, i + 1):
s += prices[j] # dispatch + unbox + box, n·w times
means.append(s / w)
# pandas: one line, one C loop over a typed column
means = prices.rolling(window=w).mean()This is not a toy example. The independent price-verification script that double-checks our Java beta engine computes rolling covariance and variance in exactly this style — two pandas one-liners doing what would be thousands of interpreted iterations:
# from verification/ipv_check.py — the real thing cov = x.rolling(window=n_returns, min_periods=2).cov(y) var = y.rolling(window=n_returns, min_periods=2).var() beta = cov / var
The full story of that script — why a Python/pandas recomputation is the right independent check on a Java engine — is in Independent verification in Python in the Financial Engineering track.
Lever two — the bigger one: a better algorithm. Vectorization shrinks the constant factor; it does not change the exponent. A vectorized rolling mean that slides a window is still O(n·w) work — a prefix-sum formulation is O(n) in anylanguage, and at scale the O(n) algorithm beats the faster constant every time (that lesson, in the beta engine's own hot path, is Prefix sums). Big-O beats micro-optimization. The lab below makes this visible.
🎛 Vectorization lab
logarithmic: 1k → 1M elements
O(n·w)
Pure Python loop
160 ms
nested loops: n windows × w adds, every one interpreted
O(n·w)
numpy vectorized (sliding window)
2.00 ms
same n·w element-ops, but inside one C loop
O(n)
O(n) prefix algorithm (any language)
315.0 µs
cumsum, shifted subtract, divide — 3 vectorized passes
numpy vs Python
80×
smaller constant factor
prefix vs numpy
6.4×
better exponent — w gone
prefix vs Python
508×
both wins combined
Estimated cost, side by side
numpy narrows the constant factor (~80× per element); the prefix algorithm changes the exponent (O(n·w) → O(n)). At large N the better algorithm beats the faster constant — in any language.
All numbers are illustrative estimates from a simple deterministic model — pure Python at ~80 ns per element-operation (interpreter dispatch, boxing and refcounting included), numpy at ~1 ns per element inside one C loop plus a fixed ~5 µs dispatch per vectorized call. Real timings vary by machine, version and data — which is exactly why rule zero is measure first. Educational tool.
Lever three: don't compute it twice — functools.lru_cache
Sometimes the fastest computation is the one you skip. One decorator turns any pure function into a memoized one, with least-recently-used eviction built in (the LRU idea itself — and why eviction matters — is its own module):
from functools import lru_cache
@lru_cache(maxsize=4096) # bounded on purpose
def discount_factor(tenor: float, rate: float) -> float:
return math.exp(-rate * tenor) # pure: same args -> same answer
discount_factor.cache_info()
# CacheInfo(hits=98120, misses=312, maxsize=4096, currsize=312)It is the right tool when the function is pure (no side effects, output depends only on arguments) and the same arguments recur. Its traps are exactly the places those conditions break:
- maxsize=None means unbounded growth — a cache that only ever gets bigger is a slow memory leak. Prefer a bound.
- Arguments must be hashable — pass a list or dict and you get a TypeError. Convert to tuple / frozenset at the boundary.
- Decorating an instance method caches on self too — every instance is pinned alive by the cache, and per-instance results interleave in one LRU. Cache a module-level function, or use functools.cached_property for per-instance values.
The memory side — and choosing the right lever
Performance is also about not holding what you don't need. A list materializes every element; a generator holds one at a time — for a million-row file that is the difference between gigabytes and kilobytes (the full treatment is Module 2). And for the objects you do hold in bulk, slots delete the per-instance __dict__:
@dataclass(slots=True) # no per-instance __dict__
class Tick:
ts: int
px: float
# ~2-3x smaller per object, faster attribute access —
# it matters when you have millions of them, not dozensThe senior decision tree, once the profile has named the hot spot:
- Is the algorithm right? Fix the exponent first — an O(n) rewrite beats any amount of tuning an O(n·w) or O(n²).
- Is it numeric array work? Vectorize with numpy/pandas — shrink the constant factor by ~100×.
- Is it repeated pure computation? lru_cache it.
- Still CPU-bound after all that? Only now consider multiprocessing — real parallelism at the cost of process overhead and serialization (why threads won't help CPU-bound work is Module 4's GIL story).
Things to try
- • In the lab, pick sum and drag N to 1,000,000 — watch the numpy bar stay a sliver while pure Python climbs into real milliseconds.
- • Switch to rolling mean — now there are three bars. Note the prefix algorithm beating vectorized numpy: same language, better exponent.
- • Drag N down to 1,000 and watch numpy's fixed dispatch overhead matter — vectorization is a large-n game.
- • In a real shell, run python -m timeit on a loop-sum vs numpy.sum of the same array and compare with the lab's illustrative model.
- • Profile something you own with cProfile — then check whether the top entry is what you would have guessed. (It won't be.)