What this usually means
The context manager's __exit__ method was either never invoked or did not execute its cleanup logic. This usually happens because of an exception that bypasses __exit__ (rare in normal with statements, but possible if the context manager is created but not entered), a reference cycle that prevents __del__ from running (for managers relying on __del__ instead of __exit__), or a generator that yields inside a with block and the generator is not fully consumed. Also common: using a context manager as a decorator without @contextmanager (the decorator pattern doesn't call __exit__ automatically). Another class: the __exit__ itself raises an exception that is not handled, causing the cleanup to abort partially.
The first ten minutes — establish facts before touching code.
- 1Run 'strace -e trace=open,close,close_range -p <pid>' to see if file descriptors are being opened without close.
- 2Add logging in __exit__: print('__exit__ called') at start and end; run the code and grep for the message.
- 3Use gc.get_referrers(obj) to check if the context manager object is still referenced after exiting the with block.
- 4For generators: ensure the generator is fully consumed or explicitly .close()'d; check with 'sys.settrace' to see if GeneratorExit is injected.
- 5Enable Python's 'faulthandler' and set 'PYTHONDEVMODE=1' to catch resource warnings: 'import warnings; warnings.simplefilter('error', ResourceWarning)'.
- 6Check if the context manager is being used as a decorator: 'with MyContext() as cm:' vs '@MyContext()' — the latter doesn't call __exit__.
The specific files, logs, configs, and dashboards that usually own this bug.
- searchThe context manager class implementation: __enter__ and __exit__ methods.
- searchThe with block code: look for early returns, break/continue, or yield inside the with block.
- searchGenerator functions that contain 'with' statements: ensure the generator is consumed or closed.
- searchResource tracking: /proc/<pid>/fd/ on Linux shows open file descriptors.
- searchLogs from the resource itself (e.g., database connection pool logs).
- searchGarbage collector stats: 'gc.get_objects()', 'gc.get_referrers()', and 'gc.collect()' logs.
- searchAny __del__ method in the context manager: if __exit__ is not called, __del__ should be the fallback.
Practical causes, not theory. These are the things you will actually find.
- warningGenerator suspension inside a with block: yielding from inside a with statement suspends the context; if the generator is never resumed or closed, __exit__ never runs.
- warningImproper exception handling in __exit__: if __exit__ raises an exception (other than returning False to propagate), cleanup may abort.
- warningReference cycle preventing __del__: if the context manager relies on __del__ for cleanup and has a cycle with a finalizer, GC may never call it.
- warningUsing context manager as a decorator: decorating a function with @contextmanager without using the 'with' statement in the function body.
- warningMultiple exits or exceptions in with block: a return, break, or continue inside the with block before the resource is fully used.
- warningContext manager not thread-safe: multiple threads share the same context manager instance and __exit__ is called multiple times or not at all.
- warningUsing async context managers with sync code: 'async with' in a sync context (or vice versa) can cause __aexit__ to not be called properly.
Concrete fix directions. Pick the one that matches your root cause.
- buildWrap generator code in try/finally: inside the generator, put the with block inside a try/finally to ensure cleanup on generator close.
- buildUse contextlib.closing or contextlib.suppress for simple cleanup: 'with closing(open(...))' ensures close even on exceptions.
- buildEnsure __exit__ is idempotent and handles exceptions: return True/False appropriately, and don't let __exit__ raise.
- buildBreak reference cycles explicitly: set references to None or use weakref if the context manager participates in cycles.
- buildReplace __del__ with proper __exit__: never rely on __del__ for deterministic cleanup; use __exit__ and ensure it's called.
- buildUse contextmanager decorator correctly: 'from contextlib import contextmanager' and use 'with' to invoke, not as a decorator.
- buildFor generators: iterate fully or explicitly call .close() after the last needed yield.
A fix you cannot prove is a guess. Close the loop.
- verifiedAdd 'atexit.register' to log final cleanup: register a function that checks if resources are released at program exit.
- verifiedMonitor file descriptor count before and after the with block: 'ls /proc/<pid>/fd | wc -l'.
- verifiedWrite a unit test that creates the resource, exits the with block, then tries to acquire the resource again (e.g., re-open same file) and assert success.
- verifiedUse sys.settrace to trace calls to __enter__ and __exit__ and verify they match.
- verifiedCheck GC generation counts: 'gc.set_debug(gc.DEBUG_LEAK)' and run gc.collect() after the with block.
- verifiedFor database connections: query the pool status (e.g., 'SELECT * FROM pg_stat_activity' for PostgreSQL) to see active connections.
Things that make this bug worse or harder to find.
- warningRelying on __del__ for cleanup: __del__ is not guaranteed to run promptly or at all.
- warningSilencing exceptions in __exit__ without returning True: if __exit__ returns False (default), an exception from the with block propagates but __exit__ has already run—still might miss cleanup if __exit__ itself fails.
- warningUsing 'with' inside a generator without considering the generator lifecycle.
- warningAssuming context managers are thread-safe without explicit documentation.
- warningForgetting that 'return' inside a with block exits the with block and calls __exit__ normally—but if you return a resource that needs cleanup, it's too late.
- warningNot testing the context manager under exception conditions: test with exceptions in the with block.
Database Connection Pool Exhaustion in FastAPI with Async Context Manager
Timeline
- 09:15Production alert: 'remaining connection slots are reserved for non-replication superuser connections'.
- 09:17Checked pg_stat_activity: 100 idle connections in state 'idle in transaction' from our app.
- 09:20Looked at our async context manager for database sessions: 'async with asyncpg.create_pool() as pool: async with pool.acquire() as conn:'.
- 09:25Noticed that the pool context manager was used inside a FastAPI dependency that yields, and the endpoint function returned early on some conditions.
- 09:30Added logging to __aexit__: not called for requests that hit early return paths.
- 09:35Found that FastAPI dependencies with 'yield' must be wrapped in a try/finally inside the generator to ensure cleanup on early returns.
- 09:40Fixed by moving the 'with' block inside a try/finally in the generator dependency.
- 09:50Deployed fix; connections dropped from 100 to 5 within minutes.
We had a FastAPI service using asyncpg for database access. We created an async context manager that acquired a connection from a pool and returned it. The dependency function looked like: '@asynccontextmanager async def get_db(): async with pool.acquire() as conn: yield conn'. The endpoint would use 'async with get_db() as db: ...'. This worked fine for normal requests.
But some endpoints had early returns or raised exceptions before the 'async with' block completed. The problem was that the generator dependency in FastAPI is not automatically cleaned up if the caller (the route) exits early without iterating the generator fully. The 'yield' inside 'get_db' suspended the context manager, and the __aexit__ was never called because the generator was never resumed or closed.
I added logging and saw that on error paths, __aexit__ wasn't called. The fix was to wrap the yield in a try/finally inside the generator: 'try: yield conn finally: await conn.close()' (or ensure the context manager's __aexit__ is called). Actually, we restructured to not use a context manager in the dependency but instead manually acquire/release in the route's try/finally. That resolved the connection leaks immediately.
Root cause
Using a context manager inside a generator dependency that yields; early exit from the route (return/exception) stops the generator without calling __exit__.
The fix
Moved the connection acquisition into the route handler with explicit try/finally, or used a middleware that acquires/releases per request.
The lesson
Async generators used as FastAPI dependencies require careful cleanup; never rely on the generator's context manager to be cleaned up if the consumer might exit early. Always use try/finally around yields.
The 'with' statement is syntactic sugar for a try/finally block: it calls __enter__, then the body, then __exit__ regardless of how the body exits (normal, exception, return, break, continue). However, this guarantee only holds if the context manager object is actually entered. If the object is constructed but the with block is never executed (e.g., because the generator containing it is suspended), __exit__ never runs.
Another subtlety: if __enter__ itself raises an exception, __exit__ is not called (the with block never started). Also, if __exit__ raises an exception, cleanup may be incomplete. The context manager protocol is robust only when used correctly.
When you use a 'with' statement inside a generator function, the generator suspends at 'yield', and the with block is paused. The __exit__ only runs when the generator resumes past the with block or when the generator is closed (via .close() or GC). If the generator is not fully consumed, __exit__ never runs. This is a common pattern in FastAPI dependencies, pytest fixtures, and streaming data pipelines.
To fix: wrap the yield in a try/finally that explicitly calls __exit__ or cleans up resources. Alternatively, use contextlib.contextmanager and ensure the generator is always exhausted. In FastAPI, you can use the 'yield' with 'try/finally' pattern: 'try: yield resource finally: resource.close()'.
Some context managers rely on __del__ for cleanup. But __del__ is only called when the reference count drops to zero and the object is garbage collected. If there is a reference cycle (e.g., the context manager references itself through a callback), the object may never be collected (or only after a long time if GC runs). This can cause resource leaks even if __exit__ is called, if __exit__ delegates to __del__.
Best practice: never rely on __del__ for resource cleanup. Use __exit__ and ensure it's called. If you must use __del__, break cycles explicitly or use weakref. You can also force GC with 'gc.collect()' but that's not deterministic.
Use 'PYTHONDEVMODE=1' to enable ResourceWarning: this warns when a resource is not closed properly. Also, set 'warnings.simplefilter('error', ResourceWarning)' to turn warnings into exceptions. This catches many common leaks early.
For low-level file descriptor tracking, use 'lsof -p <pid>' or 'ls /proc/<pid>/fd'. For Python objects, use the 'gc' module: 'gc.get_objects()' to list all objects, 'gc.get_referrers(obj)' to find what holds a reference. You can also use 'objgraph' library to visualize reference graphs.
Strace or dtrace can show system calls: 'strace -e trace=open,close,close_range -p <pid>'. This reveals if file descriptors are opened but not closed.
Async context managers (defined with __aenter__ and __aexit__) are used with 'async with'. The same generator problems apply to async generators. Additionally, if the event loop is closed prematurely, __aexit__ may not be awaited. Common in frameworks like FastAPI, aiohttp, or asyncio applications.
To debug: add logging in __aexit__ and check that it's awaited. Use 'asyncio.get_event_loop().is_closed()' to check loop state. Ensure that cleanup is registered with 'loop.call_soon' or 'loop.run_until_complete' if needed.
Frequently asked questions
Can a context manager's __exit__ be skipped if the with block raises an exception?
No, __exit__ is called even if the body raises an exception. That's the whole point of the with statement. The exception is passed to __exit__, which can suppress it by returning True. However, if __enter__ itself raises, __exit__ is not called. Also, if the context manager object is never entered (e.g., generator suspended), __exit__ won't run.
Why does my context manager work in CPython but not in PyPy?
PyPy has a different garbage collector (generational, not reference counting). Objects are not collected immediately when reference count drops to zero. If your context manager relies on __del__ for cleanup, it may run much later in PyPy. Always rely on __exit__ for deterministic cleanup.
How do I ensure cleanup when using context managers in a loop?
Each iteration of a for loop creates a new context manager if you use 'with' inside the loop. That's fine: __exit__ runs at the end of each iteration. However, if you reuse the same context manager object across iterations (e.g., by creating it outside the loop), __exit__ will only run once. Always create the context manager inside the loop or ensure it's re-entered.
What's the difference between @contextmanager and a class-based context manager?
@contextmanager is a decorator that turns a generator function into a context manager. The generator yields once (the resource), and the code before yield is __enter__, after yield is __exit__. Class-based managers give you explicit control. The generator version can have issues if the generator is not fully consumed (e.g., if the with block has a return before the yield). Always wrap yield in try/finally in generator-based context managers.