Module 6

The senior toolbelt: stdlib & libraries

What separates a senior Python engineer is rarely exotic knowledge — it's knowing which tool the language already ships, so problems that tempt juniors into fifty lines of custom code dissolve into one import. Python calls it “batteries included”; this module is the tour of the batteries worth knowing cold, then a map of the third-party world for everything else.

Half one: the stdlib you reach for daily

  • collections — the container upgrades: defaultdict(list) for grouping without key checks, Counter(words).most_common(3) for frequency counts, deque(maxlen=20) as an O(1) ring buffer.
  • itertools — lazy iteration algebra: islice(gen, 10) to take from an infinite stream, groupby(trades, key=...) for run-length grouping, accumulate(xs) for running totals — which is exactly a prefix sum, one import away.
  • functools — function algebra: lru_cache for memoization (Module 5), partial(fn, arg) to pre-fill arguments, reduce when a fold is genuinely clearer.
  • pathlib — paths as objects, not string surgery: Path("data").rglob("*.csv") replaces os.path.join + os.walk gymnastics. Prefer it over os.path everywhere.
  • dataclasses — typed value objects for free: @dataclass(frozen=True, slots=True) gives you init, repr, equality and immutability in one line.
  • enum — named constants with identity: class Side(Enum): BUY = "B"; SELL = "S" — no more stringly-typed magic values.
  • logginglogger = logging.getLogger(__name__) at module level, configure handlers once, at the entry point, never in libraries — the same rule our Java service enforces with its single Logging configuration class.
  • json / csv — the boring workhorses: json.loads / json.dumps, and csv.DictReader before you reach for pandas on a small file.
  • datetime — know aware vs naive: a naive datetime has no timezone and comparisons across zones silently lie. In finance, always aware: datetime.now(tz=ZoneInfo("America/New_York")).
  • subprocess — run external programs safely: subprocess.run([...], check=True, capture_output=True) — a list of args, never a shell string.
  • unittest.mockpatch("mymod.http_get") to fake the boundary (network, clock, filesystem) while testing your real logic.

And the de-facto test runner is not in the stdlib but belongs in this half anyway: pytest — plain assert statements with rich failure output, fixtures for setup, and parametrize to turn one test into a table of cases:

@pytest.mark.parametrize("px, qty, want", [
    (100.0, 10, 1000.0),
    (99.5,  -5, -497.5),
])
def test_notional(px, qty, want):
    assert notional(px, qty) == want   # plain assert — pytest does the rest

Half two: the library map, by domain

One line each on when a library is the right reach:

  • numpy — typed contiguous arrays and vectorized math; the moment you loop over numbers, stop and reach here.
  • pandas — labeled tables: joins, groupbys, rolling windows. It's the tool ipv_check.py uses to independently double-check the Java beta engine (that story here).
  • polars — pandas' faster columnar successor; same jobs, multicore and lazy, when pandas becomes the bottleneck.
  • requests / httpx — HTTP clients; httpx when you need async.
  • FastAPI — typed web APIs from type hints, with validation and docs generated for you — the backend serving this very site is FastAPI.
  • pydantic — runtime validation at the boundaries: parse untrusted input into typed models once, at the edge.
  • SQLAlchemy — the database layer, from raw-ish SQL core up to a full ORM.
  • matplotlib / plotly — visualization: matplotlib for static figures, plotly when you need interactivity.
  • scipy / statsmodels — numerical routines and proper statistics (regressions with p-values, not just fits).
  • scikit-learn — classical machine learning with one consistent fit/predict API.
  • pytest / ruff / mypy / black — the quality quartet: test runner, lightning-fast linter, static type checker, no-arguments formatter.
  • uv / pip / venv — environments and packaging: one venv per project, lock your deps; uv is the fast modern front-end to the same workflow.

🎛 Toolbelt matcher

Read the task, name the tool in your head, then click to check yourself.

0 / 10 revealed

Ten everyday tasks, each with the tool a senior reaches for on autopilot — and the tempting alternative that costs you lines, speed or correctness. The pattern to internalize: the right tool usually turns the task into one expression at the right layer. Educational tool.

The senior's meta-rule: dependencies must earn their keep

The instinct that ties this whole module together: prefer the stdlib until a library earns its dependency. Every dependency you add is a supply-chain liability(you now ship code you didn't write, from maintainers you don't know) and an upgrade liability (its breaking changes become your sprint work, forever). numpy earns it — you cannot write those C loops yourself. A left-pad-sized helper does not.

# the test, before every "pip install":
#   1. can the stdlib do this in < 20 lines?     -> stdlib
#   2. is the library the de-facto standard,
#      actively maintained, widely audited?      -> maybe
#   3. would I bet a 2am production page on it?  -> now decide

The beta service's Java core made exactly this call — a zero-dependency engine, with the framework kept out at the edges — and the reasoning is walked through in the Financial Engineering track. Same principle, different language.

Things to try

  • • Work through the matcher above before revealing — say your answer out loud, then check. The misses are the gaps worth closing.
  • • In a shell: from itertools import accumulate; list(accumulate([3, 1, 4, 1, 5])) — you just built the prefix-sum array from the Financial Engineering track in one call.
  • • Take a script of yours that uses os.path and port it to pathlib — count the lines that disappear.
  • • Run python -c "import this" and reread line two: “Explicit is better than implicit.” The whole toolbelt follows from it.
  • • Audit one of your projects: for each dependency in its lockfile, can you say in one sentence what it earns? If not, try deleting it.