SSerguey Asael Shinder
Java coding notes: the JVM, and writing software that lasts

Serguey Asael Shinder: A limit is a promise to give back: FastAPI's max_concurrency, Semaphore and Micronaut's leases

· by Serguey Asael Shinder / Serguey Shinder

This week three projects I follow touched the same idea from different sides: capping how many callers can hold a scarce resource at once.

FastAPI 0.163 added Depends(max_concurrency=5) for dependencies such as database sessions (docs added in #16746). Java's standard tool for this is Semaphore, and I wrote about its fine print today. Micronaut Core 5.2.24 is the other side of the coin: a release whose bug fixes are mostly about resources that were taken and not given back, such as a pooled client connection that stayed taken until its read timeout when building the request failed (#14457).

What strikes me is that the cap itself is the easy part. The hard part is the moment of return, and each of the three answers it explicitly.

FastAPI ties the return to the dependency's exit code. With yield, the session counts until the code after yield has finished, which with the default scope="request" is after the response is sent and with scope="function" is before. A dependency without yield counts only until the function returns. The framework owns both the take and the give-back, so user code cannot forget either, but it has to choose where the boundary is.

Serguey Asael Shinder: A limit is a promise to give back: FastAPI's max_concurrency, Semaphore and Micronaut's leases
A limit is a promise to give back: FastAPI's max_concurrency, Semaphore and Micronaut's leases — Serguey Asael Shinder

Java's Semaphore owns neither. Its javadoc says there is "no requirement that a thread that releases a permit must have acquired that permit", and that correct usage "is established by programming convention". The try/finally around acquire() and release() is that convention, and nothing checks it.

Micronaut's fixes show what happens when the convention lives in many places at once. The pooled connection was released on one failure path - a form encoding error - but not on the others. Listener ports were closed on stop, but only for listeners that had been published, and publication waited until all of them were bound.

The FastAPI documentation adds a warning that applies to all three: a request can deadlock if it waits for a second unit while holding the first, which it releases only when the request ends. The same is true of a thread that calls acquire() twice on a one-permit Semaphore, or of code that holds a pooled connection while waiting for another from the same pool.

The rule I take away: when I add a limit, I write the release first and put it where every exit passes through - a finally, an exit hook, a single cleanup method - and only then the acquire. If I cannot point to that one place, the limit is a number I hope for, not one I enforce.