Free-threaded Python: four Granian worker threads use one CPU core while four processes scale normally

[pythonweb2]

Hi, I am investigating how Granian behaves with CPython 3.14 free-threading and an async FastAPI/SQLAlchemy application.

The short version:

• CPython reports a free-threaded build.
• sys._is_gil_enabled() reports False.
• Granian starts four worker threads in one process.
• The application uses roughly one CPU core under load.
• The same application with four standard-GIL worker processes uses roughly 3.3 CPU cores and achieves approximately 5x the throughput.
• PostgreSQL is not the bottleneck in the free-threaded run.
• Upgrading SQLAlchemy from 2.0.52 to 2.1.0b3 did not improve the result.

I am trying to understand whether this is expected behavior for Granian’s free-threaded worker implementation, or whether there is another serialization point in the server/runtime path.

Environment

• macOS host on Apple Silicon
• Docker Desktop
• Docker engine 29.4.0
• Alpine Linux 3.24
• CPython 3.14.7
• Granian 2.8.1
• FastAPI 0.141.1
• psycopg 3.3.4
• psycopg-c 3.3.4
• PostgreSQL 18-alpine
• SQLAlchemy 2.0.52 for the original comparison
• SQLAlchemy 2.1.0b3 for the follow-up comparison
• Docker sees 12 CPUs
• No Docker CPU quota or CPU affinity was configured

The application is a small FastAPI service using async SQLAlchemy with the psycopg driver. Database engines and pools are cached per event loop because a single SQLAlchemy AsyncEngine cannot safely be shared across the separate Granian event loops.

Granian configuration

The container command is:

granian --interface asgi app.main:app

The worker count is configured through:

GRANIAN_WORKERS=4
GRANIAN_LOOP=asyncio

I did not configure additional Granian runtime threads or a separate runtime mode.

For the free-threaded run, the image was built with CPython 3.14t.

The runtime probe reported:

Python:               3.14.7
Free-threaded build:  True
GIL enabled now:      False
Distinct PIDs:        {1: 256}
Distinct threads:     {'granian-worker': 256}
Distinct event loops: 4

Granian startup also reported:

Starting granian (main PID: 1)
Spawning worker-1 with TID: 11
Spawning worker-2 with TID: 13
Spawning worker-3 with TID: 15
Spawning worker-4 with TID: 17

So this is one OS process with four Granian worker threads and four event loops.

For comparison, standard CPython 3.14 with GRANIAN_WORKERS=4 reported four separate worker PIDs:

Free-threaded build:  False
GIL enabled now:      True
Distinct PIDs:        4

Workload

The benchmark repeatedly sends:

GET /products?limit=50

The endpoint executes approximately:

rows = await db.execute(
    select(Product).order_by(Product.id).limit(50).offset(0)
)
return list(rows.scalars().all())

The load test uses:

LOGFIRE_TOKEN= LOGFIRE_SEND=false \
uv run scripts/loadtest.py \
  --path '/products?limit=50' \
  --concurrency 128 \
  --duration 20

Logfire export was explicitly disabled for all relevant runs.

The SQL query itself is very cheap:

Execution Time: 0.073 ms
Buffers: shared hit=18

The test database contains only a small number of products.

Results

Four free-threaded workers, SQLAlchemy 2.0.52

requests_ok:   6342
errors:        0
throughput:    312.2 RPS
mean latency:  407.94 ms
p50:           300.02 ms
p90:           860.99 ms
p99:           1753.98 ms
max:           4416.11 ms

Container sample during load:

API CPU:       ~102%
API memory:    182.3 MiB
Postgres CPU:  7.8% to 9.4%
Postgres memory: ~141 MiB

Idle API memory was approximately:

148.9 MiB

Four standard-GIL worker processes, SQLAlchemy 2.0.52

requests_ok:   33327
errors:        0
throughput:    1655.3 RPS
mean latency:  76.98 ms
p50:           74.83 ms
p90:           98.43 ms
p99:           129.22 ms
max:           251.02 ms

Container sample during load:

API CPU:       329.77%
API memory:    452.1 MiB
Postgres CPU:  84.38%
Postgres memory: 129.6 MiB

Idle API memory was approximately:

420.5 MiB

The four-process configuration therefore used substantially more memory, but it also used approximately 3.3 CPU cores and achieved about 5.3x the throughput.

Four free-threaded workers, SQLAlchemy 2.1.0b3

After rebuilding the image from scratch with SQLAlchemy 2.1.0b3, the runtime still reported:

Free-threaded build: True
GIL enabled now: False
Distinct PIDs: 1
Distinct event loops: 4

Two runs produced:

Run	RPS	p50	p99	Errors
1	251.5	401.85 ms	2076.95 ms	0
2	241.0	421.27 ms	2128.08 ms	0

The API remained around one CPU core during the benchmark, and PostgreSQL remained mostly idle:

API CPU:       ~105% to ~107%
Postgres CPU:  ~7% to ~8%

The SQLAlchemy 2.1 runs were not faster than the SQLAlchemy 2.0.52 run. They were somewhat slower in this particular test, although I have not treated that as a reliable SQLAlchemy regression because this is a prerelease build and the benchmark is not yet statistically rigorous.

PostgreSQL observations

The database does not appear to be the bottleneck in the free-threaded configuration.

During the four-worker free-threaded test:

Postgres CPU:       approximately 7% to 9%
Active queries:     1
Sessions:           approximately 59 to 69
max_connections:    200

The application uses:

pool_size=10
max_overflow=5

Because there is one pool per event loop, four workers can create up to:

4 * (10 + 5) = 60 connections

This explains the relatively high idle connection count, but PostgreSQL was not close to its configured connection limit.

There was one transient sample showing several idle in transaction sessions, but a follow-up query found zero such sessions after the load ended. I do not currently believe this indicates leaked transactions.

Questions

1. Is this worker topology expected for CPython free-threading in Granian 2.8.1?
2. With four free-threaded workers in one process, should independent async request handling be expected to use multiple CPU cores?
3. Are there Granian runtime settings that affect how free-threaded workers schedule asyncio event loops or execute Python code?
4. Is there a known serialization point in the free-threaded threaded server implementation that could explain the API process staying near 100% CPU?
5. Could the difference be caused by the interaction between Granian’s threaded worker model and asyncio rather than by the Python interpreter or SQLAlchemy?
6. Are there recommended diagnostics for determining whether the limitation is inside Granian, the asyncio event loop, psycopg, or SQLAlchemy?

Reproduction

I can push up my branch with all the code, lmk if that would be helpful to reproduce

[muhammadbalawal]

Four threads pinned to roughly one core while four processes scale to 3.3 is the signature of the GIL being on. Before digging into Granian internals I would confirm it is actually still off at the point where your request handlers run.

The catch is where you measured it. A free-threaded build starts with the GIL disabled, but any C extension that has not declared Py_mod_gil support turns it back on when it is imported, and that happens at import time, which is after Granian has started and done its own check. So a startup check reporting False does not tell you what is true once your application module has pulled in the whole dependency tree.

Add this inside an actual route rather than at startup:

import sys

@app.get("/gil")
async def gil():
    return {"gil_enabled": sys._is_gil_enabled()}

If that returns True, everything else is explained.

The likely candidate in your stack is psycopg-c. Compiled database drivers have been slow to declare free-threading support, and when the extension has not opted in CPython emits a RuntimeWarning along the lines of "The global interpreter lock (GIL) has been enabled to load module X, which has not declared that it can run safely without the GIL". In Docker that warning is easy to miss unless you go looking. Run with -W error::RuntimeWarning or grep the container logs for "has not declared" and you will get the exact module name.

This also fits the rest of your results. Going from SQLAlchemy 2.0.52 to 2.1.0b3 changing nothing is consistent with the driver being the thing that flips the GIL rather than the ORM. And the fact that four processes still scale fine is expected, since each process gets its own interpreter and its own GIL, so a re-enabled GIL costs you nothing in the process model and costs you everything in the thread model.

Two ways to confirm:

1. Drop the psycopg-c binary and run pure-Python psycopg for one test. If throughput scales across cores, that was it.
2. Force it with PYTHON_GIL=0 or -Xgil=0. That overrides the automatic re-enable. Treat this as a diagnostic only, since the extension is declaring that it is not safe without the GIL and you would be overriding that.

There is a closely related thread on the SQLAlchemy side at sqlalchemy/sqlalchemy#13256 covering the same "3.14t still re-enables the GIL" behaviour with this driver stack, which is worth reading.

If the in-request check comes back False, then the GIL is genuinely off and this is a real Granian scaling question rather than an environment one, and it would be worth re-testing with a trivial ASGI app that does no database work at all to separate the server from the driver.

[Yigtwxx]

On question 1, the topology is expected and it's decided more bluntly than you'd probably guess. granian/server/__init__.py is the whole thing:

from .._granian import BUILD_GIL
from .mp import MPServer as MPServer
from .mt import MTServer as MTServer

Server = MPServer if BUILD_GIL else MTServer

and BUILD_GIL comes from a compile-time cfg in the Rust extension (src/lib.rs):

#[cfg(not(Py_GIL_DISABLED))]
const BUILD_GIL: bool = true;
#[cfg(Py_GIL_DISABLED)]
const BUILD_GIL: bool = false;

So the choice is made from how the interpreter was built, at import time, and there is no runtime consultation of sys._is_gil_enabled() anywhere in the path. On a 3.14t image you get MTServer and threading.Thread(name='granian-worker', ...) unconditionally — which matches the thread name in your probe.

That matters for question 4, because it's the mechanism that turns a re-enabled GIL into the worst case rather than a wash. A C extension that hasn't declared Py_mod_gil support turns the GIL back on when it's imported, and that import happens while loading your application — after Granian has already committed to threads. Four threads plus a live GIL is one core. Under MPServer the same re-enable would have cost you nothing, since each process has its own. Granian can't currently notice the difference.

Granian itself is clear of suspicion on this, for what it's worth — the extension declares #[pymodule(gil_used = false)], so importing _granian doesn't flip anything.

The in-route sys._is_gil_enabled() check suggested above is the right first move. If it comes back True, I'd skip guessing at which dependency did it and bisect it directly, since the re-enable warning is emitted once and is easy to lose in container logs:

import sys, importlib

for name in [
    "granian", "psycopg", "psycopg_c", "psycopg_binary",
    "pydantic_core", "sqlalchemy", "fastapi",
]:
    before = sys._is_gil_enabled()
    try:
        importlib.import_module(name)
    except ImportError:
        print(f"{name:16} (not installed)")
        continue
    after = sys._is_gil_enabled()
    print(f"{name:16} {before} -> {after}{'   <-- culprit' if after and not before else ''}")

Run that as the container's entrypoint in a fresh process. The first line that transitions False -> True names the module, with no reliance on warning capture. Import order matters, so keep the drivers before the frameworks that pull them in — psycopg_c and psycopg_binary are worth listing separately since only one will be present and they're built differently. pydantic_core is on the list because FastAPI's response serialization goes through it on every one of your /products responses, so it would be squarely in the hot path if it were the one.

If the in-route check comes back False, then this is genuinely a Granian scaling question and your questions 2, 3 and 5 become the live ones. In that case I'd cut the database out entirely first — a route returning a constant dict, same load profile — because your endpoint's cost is spread across psycopg, SQLAlchemy result construction and Pydantic serialization, and any of those could be serializing independently of the server. If a constant-dict route scales across cores and /products doesn't, the server isn't the problem.

Either way, there's a reasonable feature request in here for Granian: MTServer could check sys._is_gil_enabled() after the application module is imported and log a warning, or refuse. Right now the failure is completely silent, and a 5x throughput difference with no error is a hard thing to attribute — as your write-up shows.