Module 6
The senior toolbelt: stdlib & libraries
What separates a senior Python engineer is rarely exotic knowledge — it's knowing which tool the language already ships, so problems that tempt juniors into fifty lines of custom code dissolve into one import. Python calls it “batteries included”; this module is the tour of the batteries worth knowing cold, then a map of the third-party world for everything else.
Half one: the stdlib you reach for daily
- collections — the container upgrades: defaultdict(list) for grouping without key checks, Counter(words).most_common(3) for frequency counts, deque(maxlen=20) as an O(1) ring buffer.
- itertools — lazy iteration algebra: islice(gen, 10) to take from an infinite stream, groupby(trades, key=...) for run-length grouping, accumulate(xs) for running totals — which is exactly a prefix sum, one import away.
- functools — function algebra: lru_cache for memoization (Module 5), partial(fn, arg) to pre-fill arguments, reduce when a fold is genuinely clearer.
- pathlib — paths as objects, not string surgery: Path("data").rglob("*.csv") replaces os.path.join + os.walk gymnastics. Prefer it over os.path everywhere.
- dataclasses — typed value objects for free: @dataclass(frozen=True, slots=True) gives you init, repr, equality and immutability in one line.
- enum — named constants with identity: class Side(Enum): BUY = "B"; SELL = "S" — no more stringly-typed magic values.
- logging — logger = logging.getLogger(__name__) at module level, configure handlers once, at the entry point, never in libraries — the same rule our Java service enforces with its single Logging configuration class.
- json / csv — the boring workhorses: json.loads / json.dumps, and csv.DictReader before you reach for pandas on a small file.
- datetime — know aware vs naive: a naive datetime has no timezone and comparisons across zones silently lie. In finance, always aware: datetime.now(tz=ZoneInfo("America/New_York")).
- subprocess — run external programs safely: subprocess.run([...], check=True, capture_output=True) — a list of args, never a shell string.
- unittest.mock — patch("mymod.http_get") to fake the boundary (network, clock, filesystem) while testing your real logic.
And the de-facto test runner is not in the stdlib but belongs in this half anyway: pytest — plain assert statements with rich failure output, fixtures for setup, and parametrize to turn one test into a table of cases:
@pytest.mark.parametrize("px, qty, want", [
(100.0, 10, 1000.0),
(99.5, -5, -497.5),
])
def test_notional(px, qty, want):
assert notional(px, qty) == want # plain assert — pytest does the restHalf two: the library map, by domain
One line each on when a library is the right reach:
- numpy — typed contiguous arrays and vectorized math; the moment you loop over numbers, stop and reach here.
- pandas — labeled tables: joins, groupbys, rolling windows. It's the tool ipv_check.py uses to independently double-check the Java beta engine (that story here).
- polars — pandas' faster columnar successor; same jobs, multicore and lazy, when pandas becomes the bottleneck.
- requests / httpx — HTTP clients; httpx when you need async.
- FastAPI — typed web APIs from type hints, with validation and docs generated for you — the backend serving this very site is FastAPI.
- pydantic — runtime validation at the boundaries: parse untrusted input into typed models once, at the edge.
- SQLAlchemy — the database layer, from raw-ish SQL core up to a full ORM.
- matplotlib / plotly — visualization: matplotlib for static figures, plotly when you need interactivity.
- scipy / statsmodels — numerical routines and proper statistics (regressions with p-values, not just fits).
- scikit-learn — classical machine learning with one consistent fit/predict API.
- pytest / ruff / mypy / black — the quality quartet: test runner, lightning-fast linter, static type checker, no-arguments formatter.
- uv / pip / venv — environments and packaging: one venv per project, lock your deps; uv is the fast modern front-end to the same workflow.
🎛 Toolbelt matcher
Read the task, name the tool in your head, then click to check yourself.
Ten everyday tasks, each with the tool a senior reaches for on autopilot — and the tempting alternative that costs you lines, speed or correctness. The pattern to internalize: the right tool usually turns the task into one expression at the right layer. Educational tool.
The senior's meta-rule: dependencies must earn their keep
The instinct that ties this whole module together: prefer the stdlib until a library earns its dependency. Every dependency you add is a supply-chain liability(you now ship code you didn't write, from maintainers you don't know) and an upgrade liability (its breaking changes become your sprint work, forever). numpy earns it — you cannot write those C loops yourself. A left-pad-sized helper does not.
# the test, before every "pip install": # 1. can the stdlib do this in < 20 lines? -> stdlib # 2. is the library the de-facto standard, # actively maintained, widely audited? -> maybe # 3. would I bet a 2am production page on it? -> now decide
The beta service's Java core made exactly this call — a zero-dependency engine, with the framework kept out at the edges — and the reasoning is walked through in the Financial Engineering track. Same principle, different language.
Things to try
- • Work through the matcher above before revealing — say your answer out loud, then check. The misses are the gaps worth closing.
- • In a shell: from itertools import accumulate; list(accumulate([3, 1, 4, 1, 5])) — you just built the prefix-sum array from the Financial Engineering track in one call.
- • Take a script of yours that uses os.path and port it to pathlib — count the lines that disappear.
- • Run python -c "import this" and reread line two: “Explicit is better than implicit.” The whole toolbelt follows from it.
- • Audit one of your projects: for each dependency in its lockfile, can you say in one sentence what it earns? If not, try deleting it.