Module 4

The GIL & concurrency

CPython has one Global Interpreter Lock: only the thread holding it may execute Python bytecode, so no matter how many threads you start, one of them runs Python at any instant. Spin up four threads to crunch numbers in pure Python and you get four threads taking turns — the same total work, plus lock traffic. Threads do not parallelize CPU-bound Python.

But here's the half of the story that makes threads useful anyway: the GIL is released during blocking I/O — waiting on a socket, a file, a database — and inside well-behaved C extensions (numpy releases it around its heavy kernels). A thread that is waitingisn't holding the lock, so ten threads can all wait at once. Threads parallelize waiting.That single sentence decides most concurrency choices you'll ever make in Python.

The decision table

Workload	Reach for	Why
I/O-bound, moderate concurrency	ThreadPoolExecutor	GIL released while blocked — threads overlap the waits, shared memory, no ceremony.
CPU-bound Python	multiprocessing	Separate interpreters, separate GILs — real parallelism, at the cost of startup and serializing arguments/results.
Massive I/O concurrency (10k sockets)	asyncio	One event loop, cooperative await — a waiting connection costs a heap object, not an OS thread.
C-extension numerics (numpy)	threads are fine	numpy releases the GIL inside its kernels, so threads genuinely run the math in parallel.

Real code: hammering a service with threads

This is the actual load test from the beta service's verification harness — hundreds of HTTP requests fired at a running server. Each request is almost pure waiting (the server does the work), which is exactly the threads sweet spot:

from concurrent.futures import ThreadPoolExecutor

def fetch(url: str):
    start = time.perf_counter()
    with urllib.request.urlopen(url, timeout=120) as response:
        body = response.read()          # GIL released while blocked here
    return (time.perf_counter() - start) * 1000, body

# phase 1: cold stampede — 60 never-seen pairs requested at once
urls = [beta_url(t, b, 252) for t, b in pairs]
wall_start = time.perf_counter()
with ThreadPoolExecutor(args.workers) as pool:
    cold = list(pool.map(lambda u: fetch(u), urls))
cold_wall = time.perf_counter() - wall_start

Sixteen worker threads, sixteen requests in flight: while each thread sits in urlopen waiting for bytes, it holds no GIL, so the others proceed. Wall-clock drops by roughly the worker count even though only one thread ever executes Python at a time. The with block is doing quiet work too — the executor shuts down and joins its threads on exit, and pool.map re-raises any worker exception in the caller instead of swallowing it.

🎛 GIL simulator

Workload:Strategy:

Tasks (100 ms nominal each)8

0 ms850 ms

solid = executing Python bytecode (holding the GIL) · translucent = blocked on I/O (GIL released) · gray = process startup · dashed line = finish

Wall-clock

800 ms

8 tasks × 100 ms of work

Speedup

1.0×

vs single thread (800 ms)

Why

The GIL admits one thread at a time — slices interleave, but wall-clock equals the single-thread total.

Times are an illustrative, deterministic simulation of the scheduling model — not measurements. Real numbers depend on task size, I/O latency, how much a C extension releases the GIL, and (for processes) how expensive your arguments are to serialize. The shapes, though, are the ones you will see in a real profiler.

asyncio in two paragraphs

async def defines a coroutine; await marks the points where it may pause. A single-threaded event loop runs thousands of coroutines by switching between them at those await points — when one coroutine awaits a socket, the loop runs another. Concurrency is cooperative: nothing is preempted, so there are no lock-ordering surprises mid-statement, and an idle connection costs a small object on the heap rather than an OS thread. That's how one process holds 10,000 open sockets.

The costs: contagion — await only works inside async def, so async spreads all the way down your call stack, and your libraries (HTTP client, DB driver) must be async-aware or you're blocking the loop anyway. And when not to use it: CPU-bound work. A coroutine yields only at await; a numeric loop never awaits, so it starves every other coroutine until it returns. For a few hundred concurrent requests, a ThreadPoolExecutor is usually the simpler, equally fast answer — asyncio earns its complexity at the extreme fan-out end.

The senior rules

Never share mutable state across threads without a threading.Lock. The GIL makes single bytecodes atomic, not your logic — a check-then-act like if key not in cache: cache[key] = build() can interleave between the check and the act.
Prefer queue.Queue for handoff. A thread-safe queue between producer and consumer threads removes most reasons to share structures at all.
Know what's coming, build for what's here.Python 3.13+ ships an experimental free-threaded (no-GIL) build — but the interpreter you deploy on today has the GIL, and the model above is today's reality.

Things to try

• CPU-bound + 4 threads — the bars interleave across lanes, but wall-clock equals the single thread. That's the GIL in one picture.
• Same workload, 4 processes — four lanes truly overlap; speedup near 4×, minus the gray startup segments.
• I/O-bound + 4 threads — translucent waits stack in parallel because a blocked thread holds no GIL.
• I/O-bound + asyncio — one lane, interleaved slices, same wall-clock as the thread pool.
• CPU-bound + asyncio — no speedup at all: nothing ever awaits, so the loop can't interleave.

← Module 3: Typing & dataclasses Module 5: Performance & profiling →