Python: independent verification & load testing

Hammering it: load_test.py

Correct numbers under one request prove little about correct numbers under sixteen concurrent ones. The load harness attacks the running service with a thread pool, timing every call from the outside (from verification/load_test.py):

def fetch(url: str):
    start = time.perf_counter()
    with urllib.request.urlopen(url, timeout=120) as response:
        body = response.read()
    return (time.perf_counter() - start) * 1000, body

# ---- phase 1: cold stampede -------------------------------------------
urls = [beta_url(t, b, 252) for t, b in pairs]
with ThreadPoolExecutor(args.workers) as pool:
    cold = list(pool.map(lambda u: fetch(u), urls))

Three phases, each aimed at a specific failure mode. Phase 1 (cold stampede) fires dozens of never-seen ticker pairs at once, exercising the synchronized pair-cache build path — the server-side tookMs field lets the script verify that each pair's cache was built exactly once rather than redundantly per thread. Phase 2 (warm throughput) sends hundreds of requests with mixed window sizes and reports throughput plus wall-clock p50 / p95 / p99 latencies alongside the server's own timing. Phase 3 (consistency) is the sneaky one — 64 threads issue the identical query and every response body must be byte-identical:

url = beta_url(*pairs[0], 252)
with ThreadPoolExecutor(args.workers) as pool:
    bodies = [b for _, b in pool.map(lambda u: fetch(u), [url] * 64)]
# tookMs varies per request; compare everything before it
canon = {b[: b.rfind(b'"tookMs"')] for b in bodies}
consistent = len(canon) == 1

If concurrency ever corrupted shared state — a half-built cache, a race in the prefix sums — two of those 64 bodies would differ, and the set would have more than one element. One line of Python, and an entire class of race condition has nowhere to hide.

Manufacturing the data: known betas and deliberate defects

A load test needs volume: make_load_data.py generates 100 tickers × 40 years of weekday closes(about a million rows). But it doesn't generate random noise — it uses a one-factor model, so every ticker's beta against the MKT baseline is known by construction and the computed betas are meaningful, not just numerically valid (from verification/make_load_data.py):

# market factor returns, then each ticker = beta * market + idiosyncratic
market = [rng.gauss(0.0003, 0.010) for _ in days]
...
true_beta = 1.0 if name == "MKT" else rng.uniform(0.3, 2.2)
sigma = 0.0 if name == "MKT" else rng.uniform(0.005, 0.02)
for i, day in enumerate(days):
    ret = true_beta * market[i] + (rng.gauss(0.0, sigma) if sigma else 0.0)
    price *= math.exp(ret)

Its sibling, make_dirty_sample.py, does the opposite job: it takes real (unadjusted) Yahoo Finance closes — genuine stock splits and all — and injects specific data-quality defectsso the service's quality analyzer has something real to find:

# stale run: freeze 10 consecutive JPM closes starting 2021-06-01
rows = [(d, t, frozen if t == "JPM" and d in stale_dates else c) for d, t, c in rows]

# calendar gap: SPY loses March 2023 entirely
rows = [r for r in rows if not (r[1] == "SPY" and r[0].startswith("2023-03"))]

# duplicates: repeat five MSFT rows verbatim
rows.extend(msft[100:105])

# single-row ticker
rows.append(("2020-01-02", "GHOST", 12.34))

Duplicates, calendar gaps, stale runs, insufficient history — plus three unparseable rows (garbage text, an invalid date, a negative price) spliced in at fixed positions. Each defect exists to prove one specific detector fires. Test data isn't found; it's engineered.