Python ConnectionResetError: distinguish peer reset from local shutdown

The symptoms

• •Traceback ends with ConnectionResetError: [Errno 104] Connection reset by peer while a long-lived socket, HTTP keep-alive session, or websocket was idle or mid-transfer.
• •Windows-only traceback reports ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host, often from asyncio, requests, urllib3, or socket.recv calls.
• •The error appears immediately after the server closes a session due to idle timeout, TLS renegotiation, or graceful shutdown, and the client never observes a clean close handshake.
• •Production logs show ConnectionResetError only on the first request after a process restart, on the first request after a long pause, or only when traffic crosses a specific load balancer, NAT, or proxy hop.
• •Retry loops that swallow ConnectionResetError as a generic IOError keep failing at the same rate, indicating the close is not transient on the application side.

Likely causes

• •The remote peer sent a TCP RST segment because its socket had already been closed, its application called abortive close, or its keepalive threshold was exceeded before the client read.
• •A stateful network element (load balancer, NAT gateway, sidecar proxy, cloud egress) silently evicted the flow from its translation table and replied with RST to the next client packet.
• •The local process closed or replaced its socket handle (process restart, fd reuse, server bind to the same port) while the peer still held the old four-tuple open and sent data into it.
• •The local stack tore the connection down during graceful shutdown, interpreter teardown, or signal handling, and the peer saw a RST instead of a FIN because the socket buffer was discarded.
• •TLS layer errors (certificate validation failure, protocol version mismatch, ALPN negotiation abort) caused the underlying TCP socket to be reset before the application saw a clean SSL error.

First ten minutes

1. 01Capture errno from the exception (e.g. except ConnectionResetError as e: e.errno, e.strerror) and record whether the value is 104 on POSIX or 10054 on Windows, which is the first discriminator between RST-shaped close and other IO errors.
2. 02Record the local and remote socket address associated with the failure (sock.getpeername(), sock.getsockname()) at the moment of the exception so the four-tuple can be matched against server access logs.
3. 03Mark whether the failing call was a read (recv, read), a write (send, write), or a higher-level operation, because ECONNRESET surfaces most reliably on the next read after a peer-side abort.
4. 04Compare the timestamp of the reset to the server's last successful response on the same connection; a reset that arrives immediately after a 200 response suggests a server-side abort rather than a mid-stream timeout.
5. 05Check whether the error correlates with process restart, deployment, configuration reload, or signal receipt on the client side, which indicates local shutdown rather than remote RST.

Evidence to collect

• •The exact exception class and errno value (104 / WSAECONNRESET 10054) plus the OS string returned, which is the minimum signal that a RST was received.
• •The peer address and local address of the failing socket, plus the connection's start time and last successful I/O time, so the flow can be located in server or proxy logs.
• •Server-side records on the same peer IP and port showing whether the close was a clean FIN, an abortive close, or an idle-timeout eviction during the relevant window.
• •Network element logs (load balancer, NAT, proxy) for the same five-tuple showing session age, bytes transferred, and whether the element issued a RST.
• •Application-side signals (SIGTERM, SIGHUP, interpreter shutdown, atexit handlers) active within seconds of the failure, which implicate local shutdown rather than peer reset.

Where to look

• •The Python socket boundary: the syscall that raised the error, and the exception's errno attribute, which is the only authoritative signal of a received RST.
• •The remote socket boundary: the server's access log, application log, and close-path code on the matching peer IP and source port, to see whether the server issued FIN or RST.
• •The middlebox boundary: load balancer connection tables, NAT translation timeouts, and sidecar proxy session logs, since RSTs that originate there will not appear in the server's log.
• •The TLS boundary: TLS alert records and certificate validation errors above the socket, because a TLS-layer abort can surface as ConnectionResetError on the read path.
• •The process boundary: client shutdown sequence, file descriptor reuse, and graceful-stop handlers, where local closes can be misclassified as peer resets.

Diagnostic steps

1. 01Branch on the exception's errno: if errno is 104 (POSIX) or WSAECONNRESET (Windows), classify the event as a received TCP RST until evidence contradicts it; if errno is EBADF, ENOTCONN, or EPIPE, the cause is local and a different diagnosis applies.
2. 02Correlate the peer address and source port against the remote server's connection log in the failure window to determine whether the server recorded a clean close, an error close, or no close at all.
3. 03If the server has no record of the close, inspect the load balancer or proxy in front of it for an evicted session; a middlebox RST will not appear in either endpoint's application log.
4. 04If the server records an abortive close or application error at the same timestamp, attribute the RST to the peer application and treat it as a peer-reset, not a network failure.
5. 05If the client received the error during shutdown, signal handling, or fd reuse, classify the event as a local shutdown that the peer translated into a RST, and fix the close path on the client side.
6. 06If the error only occurs inside a TLS handshake or renegotiation, capture the TLS alert from the layer above the socket and treat the RST as a symptom, not the root cause.

Common mistakes

• •Assuming every ConnectionResetError is a transient network blip and adding a retry loop, which silently multiplies load on a peer that has already decided to drop the session.
• •Catching ConnectionResetError as a bare Exception or OSError without inspecting errno, which conflates peer RST with local EBADF and EPIPE and hides the real boundary.
• •Reading the error as 'connection refused' and looking for an unbound port, when ECONNRESET specifically means a previously established connection was terminated, not that a connect() failed.
• •Ignoring the local shutdown path and assuming the peer is at fault, even when the reset only appears during the client's own restart, reload, or signal handling.
• •Trusting client-side logs alone and skipping server-side correlation, which leaves middlebox resets invisible because the server's connection table will not show them.

Safe fixes

• •If evidence shows a peer RST from a server-side abort, fix the server's close path to send a clean FIN and drain its write buffer, then add a client-side retry that gates on the confirmed errno rather than on any IOError.
• •If evidence shows a middlebox eviction, raise the idle timeout on the load balancer or NAT to exceed the longest expected gap between client requests, or move to a protocol that tolerates eviction (HTTP/2 with GOAWAY handling).
• •If evidence shows a local shutdown leaking RSTs, replace abrupt socket closes with shutdown(SHUT_RDWR) followed by a bounded drain and a context-managed close, so the peer observes FIN instead of RST.
• •If the error originates inside a TLS handshake, surface the TLS alert above the socket so the application can log a specific cause rather than masking it as ConnectionResetError.
• •Add a single, narrow except ConnectionResetError handler that records errno, peer address, and the failing operation, so future occurrences can be classified automatically rather than re-diagnosed by hand.

Prove the fix

1. 01Repeat the original connection workflow across the same peer and network path and confirm the reset no longer occurs, or is classified as an expected peer-close event.
2. 02Confirm the client releases its socket and retry state after a reset, then completes a later request without a leaked connection or duplicate side effect.
3. 03Run the regression test with a controlled peer close and a normal response, verifying that the two cases produce distinct, documented outcomes.

Prevention and next steps

• •Standardize on a single close path in client code: shutdown(SHUT_RDWR) with a bounded drain inside a context manager, so peers always see FIN and never an unannounced RST.
• •Keep idle and read timeouts on the server shorter than any middlebox eviction timeout on the path, so the server closes cleanly before a NAT or load balancer can RST the flow.
• •Tag every ConnectionResetError catch site with the errno and the failing operation, so the next incident can be classified as peer-reset or local-shutdown without re-reading the traceback.
• •Monitor the rate of errno-104 and WSAECONNRESET errors per peer IP and per deployment window, so a spike tied to a release can be attributed to a local shutdown change rather than the network.

Safe commands and checks

python -c "import socket,errno; s=socket.socket(); print('ECONNRESET',errno.ECONNRESET)"
python -c "import errno; print('WSAECONNRESET',getattr(errno,'WSAECONNRESET','n/a'))"
python -m py_compile <path_to_app_module>
python -c "import socket; help(socket.socket.close)" | head -n 40