API 502 diagnosis checklist

The symptoms

• •A client receives HTTP 502 Bad Gateway from an intermediate (gateway, reverse proxy, CDN edge, API gateway, ingress) rather than from the origin application directly.
• •Response headers reveal a non-origin server identity such as nginx, Envoy, HAProxy, AWS ALB, Cloudflare, or a managed API gateway, while the upstream application framework is absent.
• •The 502 correlates with a clean error body but no application stack trace, indicating the gateway never received a parseable reply from upstream.
• •The same client request succeeds when retried directly against the origin, but fails through the gateway path, isolating the boundary.
• •502 frequency rises during upstream deploys, cold starts, certificate rotations, or DNS changes that touch the origin but not the gateway configuration.

Likely causes

• •Upstream connection refusal: the gateway cannot open a TCP connection to the origin because the origin process is down, the listener is bound only to loopback, or a security group/host firewall blocks the gateway's source IP.
• •Upstream connection timeout: the gateway establishes TCP but the origin does not complete TLS, send HTTP headers, or stream a response within the configured gateway read timeout.
• •Protocol or header parse failure: the origin sends a malformed response, an invalid Content-Length, a chunked-encoding violation, or HTTP/2 framing that the gateway cannot decode, so it surfaces 502 instead of forwarding.
• •TLS handshake failure on the upstream hop: expired or rotated origin certificate, SNI mismatch, or cipher incompatibility on the gateway-to-origin channel, distinct from client-to-gateway TLS.
• •Origin returns 5xx that the gateway reclassifies: a 500/503/504 from upstream that the gateway translates into a 502 to the client because of its own error-mapping policy.
• •DNS resolution failure on the gateway: the gateway's resolver cannot resolve the upstream hostname after a TTL expiry or zone cutover, producing an empty reply and a 502.

First ten minutes

1. 01Capture the exact 502 response: status line, full header set (especially Server, Via, X-*, and any proxy-added correlation IDs), body, and the request that produced it. The MDN status reference defines 502 as received from a server acting as gateway or proxy when an invalid response is received from upstream.
2. 02Identify the gateway identity from the Server or Via header and confirm which hop is generating the 502; do not assume the origin is the responder.
3. 03Issue the same request directly to the origin host (bypassing the gateway DNS or path) and observe whether a non-502 reply is returned; this single comparison separates gateway faults from origin faults.
4. 04Pull the gateway's access and error logs for the request ID or time window and inspect the upstream connect, read, and write timings the gateway recorded for that transaction.
5. 05Check upstream liveness from the gateway's network namespace with a TCP-only probe to the origin host and port; success here rules out connection refusal while isolating protocol-layer issues.

Evidence to collect

• •The exact 502 response line and headers, including Server, Via, X-Request-ID, X-Trace-ID, X-Correlation-ID, and any hop-by-hop headers that name the responder.
• •Gateway access log entry for the failing request, including upstream address, upstream connect time, upstream header time, upstream response time, and upstream status (which is often -1 or blank on a true bad-gateway).
• •Gateway error log entry for the matching time, capturing the specific reason text the gateway emits (connect failed, timeout, SSL handshake failed, invalid header, upstream prematurely closed connection).
• •Origin-side access and error logs for the same request ID and window to determine whether the origin ever saw the request and what it returned.
• •DNS resolution results from the gateway's resolver for the upstream hostname, including TTL and record type, to detect stale or missing records.
• •TLS certificate metadata for the origin (subject, issuer, validity window, SAN list) seen from the gateway's perspective, not from the public client perspective.

Where to look

• •The gateway-to-origin network boundary: TCP reachability, MTU, intermediate load balancers, security groups, and host firewalls that may drop or reset packets from the gateway source.
• •The TLS termination hop on the gateway-to-origin channel, distinct from the client-facing TLS termination, including SNI the gateway sends and the certificate chain the origin presents.
• •The gateway's upstream configuration: timeouts (connect, send, read, keepalive), retry and buffer policies, and the protocol version the gateway negotiates with the origin (HTTP/1.1 vs HTTP/2).
• •The origin application layer: process state, listener bind address, request-handler errors, and any dependency that may stall responses past the gateway's read timeout.
• •The DNS resolver the gateway uses for the upstream hostname, including any split-horizon or private-zone configuration that may differ from public resolution.
• •Certificate and key material stores used by the gateway for the origin, including rotation timestamps and any intermediate or cross-signed chains.

Diagnostic steps

1. 01Confirm the responder: parse Server and Via headers on the 502 to identify the gateway software and version. A 502 served by the origin framework itself is essentially impossible; if no proxy identity appears, the origin is fronted by an invisible hop that must be found.
2. 02Classify the failure layer by gateway log reason: connect failure points to reachability or listener state; connect timeout points to firewall drop or origin hang; read timeout points to slow origin or undersized gateway read window; SSL handshake failed points to certificate or cipher issues; invalid response points to protocol or framing mismatch.
3. 03Repeat the same request directly against the origin's published endpoint. If the origin returns a healthy response, the fault lies on the gateway-to-origin path. If the origin also returns a 5xx, the gateway is faithfully relaying an origin problem and the gateway's own 502 may be a reclassification.
4. 04Compare DNS results between the gateway's resolver and an external resolver for the upstream hostname. A divergence indicates split-horizon or stale-cache issues that produce a 502 even when TCP otherwise works.
5. 05Validate the gateway-to-origin TLS chain independently of the client-facing chain, using the gateway's SNI and the origin's certificate, to detect expired or rotated material on the upstream hop.
6. 06Measure upstream timing from the gateway logs: upstream connect time, upstream header time, and upstream response time. A header time that approaches the configured read timeout isolates the slow-response class of 502.
7. 07Check origin resource state (CPU, memory, event loop saturation, connection pool exhaustion, GC pauses) during the 502 window, since origin-side stalls commonly surface as gateway read timeouts and then 502s.

Common mistakes

• •Treating the 502 as an origin bug and restarting the application before confirming the gateway even reached it; gateway connect and read timing distinguish a never-connected 502 from a slow-origin 502.
• •Reading the client-facing TLS certificate instead of the origin certificate seen by the gateway; the gateway-to-origin chain is a separate trust path with its own expiry clock.
• •Conflating a 502 with a 504: 504 is Gateway Timeout (the upstream did not respond in time), while 502 is Bad Gateway (the upstream response was invalid, malformed, or unreachable). The MDN definitions make the distinction explicit and the gateway log reason confirms it.
• •Rotating origin certificates without updating the gateway's trust store, producing a fresh batch of 502s that look like outages but are TLS handshake failures on the upstream hop.
• •Assuming the gateway's upstream DNS cache is in sync with authoritative DNS after a hostname change, producing 502s that resolve cleanly from outside the gateway.
• •Bumping gateway timeouts upward as a first response without measuring where time is actually spent; this masks slow-origin bugs and converts 502s into prolonged client waits.

Safe fixes

• •If gateway logs show upstream connect failure: confirm the origin listener is bound to the interface the gateway connects to, and that the firewall permits the gateway's source IP on the origin port; apply only after the bind address and rule state are verified.
• •If gateway logs show upstream read timeout: raise the gateway's read/proxy_read_timeout only after measuring upstream header time and confirming the origin is genuinely slow, and only to a value justified by the origin's p99 response time, not as an unbounded increase.
• •If gateway logs show SSL handshake failed on the upstream hop: replace the origin certificate with one whose SAN covers the gateway's SNI and whose chain is trusted by the gateway's store, then re-test the gateway-to-origin handshake explicitly.
• •If gateway logs show invalid response from upstream: stop the gateway from receiving malformed framing by correcting the origin's Content-Length or chunked encoding, and verify with a captured upstream response that parses cleanly.
• •If DNS resolution from the gateway diverges from authoritative: force a controlled cache flush on the gateway and reduce the upstream record TTL only if the gateway supports honoring it, after confirming the divergence rather than guessing.
• •If the gateway is reclassifying an origin 5xx into a 502: correct the origin behavior so the original status is meaningful, and keep the gateway's status passthrough enabled unless a deliberate reclassification policy is documented.

Prove the fix

1. 01Send a known-good request through the full gateway path and observe a non-502 response with the expected status, headers, and body; repeat across at least one representative sample per route or upstream cluster.
2. 02Inspect the gateway access log for that request and confirm upstream_status is the real origin status (not -1 or blank) and upstream_response_time is within the origin's healthy range, evidencing that the gateway completed the upstream exchange rather than substituting a 502.
3. 03Run a sustained probe (a few minutes at a realistic client rate) and observe zero 502 responses from the gateway over the window, with no gateway error-log entries naming the previously diagnosed failure reason.
4. 04Capture a second 502 from a synthetic failure injection (origin stopped, certificate removed, or DNS NXDOMAIN simulated in a staging environment) and confirm the gateway log reason matches the injection; this proves the classifier still works after the fix and that the production absence of 502 is not a logging regression.
5. 05Verify the client-facing Server and Via headers on the healthy response still identify the gateway, confirming the fix did not bypass the proxy layer.

Prevention and next steps

• •Instrument the gateway to emit upstream connect, header, and response timings on every request, and alert on upstream_status missing or equal to -1 as a leading indicator of 502s.
• •Treat the gateway-to-origin TLS chain as a first-class dependency with its own expiry monitoring, separate from client-facing certificates, and automate rotation with overlap so the gateway never trusts a missing intermediate.
• •Maintain gateway timeouts that are justified by the origin's measured p99 response time, reviewed whenever the origin's latency profile changes, so 502s are not masked by oversized buffers.
• •Use a synthetic external probe that bypasses the gateway to verify origin health independently, so that gateway-origin faults and origin-internal faults are not conflated in incident review.
• •Document the status passthrough policy of the gateway so that a 502 cannot silently mask an origin 5xx, and so that reclassification is visible to operators rather than implicit.

Safe commands and checks

openssl s_client -connect <origin-host>:<origin-port> -servername <origin-sni> -showcerts </origin-sni></origin-host></origin-port>
openssl x509 -in <origin-cert.pem> -noout -subject -issuer -dates -ext subjectAltName
dig +short <origin-host> @<authoritative-resolver>