03 / Symptom to cause
Begin with what users and operators can actually see.
Diagnostic paths for intermittent errors, stuck work, unreachable deployments, stale data, and local-versus-remote mismatches.
API intermittently returns 502: trace the proxy boundary
Diagnose intermittent HTTP 502 responses at the proxy boundary by distinguishing upstream silence, upstream crash, and proxy-side framing mismatches. The guide treats 502 as a Bad Gateway signal that the proxy synthesized because it could not assemble a valid response from the origin, and walks from request-shape inspection to upstream socket evidence before any configuration change.
Open guide →Background job remains running forever: find the missing completion edge
A symptom-first guide for engineers whose queue worker starts a job but never reports it as completed or failed. It focuses on the missing completion edge: the place in the worker lifecycle where the job is supposed to transition from active to a terminal state, and where the handler silently fails to call it.
Open guide →Cache appears updated but users see old data
When an engineer has confirmed that a cache write succeeded, yet end users continue to observe the previous value, the issue is almost always a routing or layering problem rather than a failed write. The cache may be perfectly updated on one node, key, or layer, but the reader is hitting a different replica, a stale sibling cache, or an HTTP/browser cache that sits in front of the application.
Open guide →CI passes locally but fails remotely: compare execution context
Diagnoses cases where a build or test suite succeeds on a developer's machine yet fails on a remote CI runner. The core failure mode is a mismatch between the two execution contexts: dependencies, permissions, environment variables, file layout, timing, or OS-level behavior. The guide walks from first symptom (a red CI run with green local tests) to verification that the remote environment has been aligned with the local one, using only the workflow file as the source of truth.
Open guide →API returns 429 only during bursts: calculate the request shape
Diagnose 429 Too Many Requests responses that appear only during traffic bursts rather than steady-state load. The guide frames the problem as a request-shape mismatch with the provider's rate policy: a client that stays under per-second limits at idle can still violate burst, concurrency, or token-bucket ceilings once traffic shape changes. Engineers learn to capture the burst envelope, compare it against documented and observed provider limits, and verify the fix by reproducing the same burst shape without the rejection.
Open guide →API returns 502 in one region: compare regional upstream paths
When an HTTP API returns 502 in only one region while other regions serve correctly, the fault is scoped to that region's path to upstream. A 502 from a gateway or CDN means an upstream in the regional chain returned an invalid or unreachable response, so the work is to compare the failing region's edge, routing, and upstream hops against a healthy region. Treat the symptom as a regional path divergence, not as a generic backend bug, and isolate which segment (DNS, TLS, origin, WAF, health check) is responsible before changing anything.
Open guide →API is slow after idle periods: investigate connection warm-up
Idle-slow APIs are a classic warm-up problem: the first request after a quiet window pays for connection setup, pool growth, JIT compilation, or downstream dependency wake-up, and only subsequent calls return to a steady-state latency. This guide walks engineers through a conservative triage to prove the warm-up hypothesis, distinguish it from genuine capacity or code regressions, and apply safe, evidence-conditional mitigations.
Open guide →API times out only for large payloads: locate upload and processing deadlines
Diagnose the case where an HTTP API returns timeout or 502 errors only when the request body grows past a threshold, by separating upload, buffering, parsing, and downstream-processing deadlines. The guide frames large payloads as a deadline-stickiness problem: each hop in the path has its own timer, and larger bodies consume more of every timer before the application can return a result.
Open guide →Browser input lags while CPU is moderate: locate long main-thread tasks
When a web page's input feels sluggish despite a moderate CPU profile, the cause is usually long main-thread tasks blocking the event loop rather than sustained CPU saturation. This guide walks engineers through using the Performance API and browser performance panels to identify which scripts, layout work, or style recalculations are starving input handlers, then narrows the diagnosis to actionable fixes.
Open guide →Browser memory climbs after navigation: find objects surviving route changes
Diagnose climbing browser memory after SPA route navigation by separating routing artifacts from real object retention. The guide frames every symptom around an observable measurement, every fix around a release checkpoint, and treats retained-but-unreachable state as the working hypothesis before any speculative cause is accepted.
Open guide →CI times out while local tests pass: compare runner capacity and waiting resources
Diagnose CI timeouts that do not reproduce locally by comparing runner capacity, shared service latency, and cleanup paths. This guide focuses on the asymmetric failure mode where developer machines finish in minutes but hosted runners stall on slow I/O, contended services, or dangling background work.
Open guide →CSRF validation fails after a tab sits idle: trace token lifetime and refresh
CSRF validation failures that appear only after a browser tab sits idle are almost always caused by the anti-CSRF token embedded in a previously rendered form expiring, rotating, or being scoped to a session that the server no longer recognizes. This guide walks through tracing token lifetime, identifying where the mismatch is created, and refreshing the token safely without disturbing unrelated state.
Open guide →CSRF validation fails in the browser but not a script: compare cookies and origin
Explains why CSRF validation succeeds for scripted clients but fails for real browsers, framed as a divergence in what request evidence (cookies, Origin/Referer, headers) each client presents. The guide treats the browser as a boundary that silently edits cookies and origin metadata, and gives an evidence-first triage to identify which boundary actually caused the rejection.
Open guide →Deployment is green but health checks fail: separate build success from serving
A deployment reports a green build/pipeline status while the application's health endpoint returns failure or never becomes reachable. The guide separates artifact creation from runtime serving, shows how to read evidence at the build, network, and process boundaries, and gives conditional remediation tied to observable checks rather than pipeline color.
Open guide →Distributed lock contention spikes: find the widened critical section
A operational playbook for engineers who suspect a distributed lock (e.g., Redis-backed) is causing elevated contention because the protected critical section has grown. Covers how to distinguish lock-queue buildup from external latency, how to measure hold time and acquisition frequency, and how to prove that a recent code change widened the guarded block before changing the lock strategy.
Open guide →Distributed lock never releases: compare lease ownership and expiration
A distributed lock that never releases is a classic coordination failure: the lock record outlives its owner, or a renewal path keeps it artificially alive past the intended critical section. This playbook walks through how to compare lease ownership metadata against expiration state in a running system, decide whether the lock is genuinely orphaned or being kept warm by a faulty renewer, and apply conditional remediation without disrupting healthy holders.
Open guide →Docker service works in container but not from host: trace published address
When a Docker container reports a service listening inside its namespace but the host cannot reach the published port, the failure almost always lives in the publication path between the container's network namespace and the host's listener, not in the application itself. This guide frames the bug as a binding-vs-publication discrepancy and walks engineers from observable symptom to a verified fix using only read-only Docker commands and generic reachability checks.
Open guide →Docker volume data disappears after restart: compare mount and container paths
When files written inside a Docker container vanish after a restart, the cause is almost always that the writes landed on an ephemeral layer or on a different mount than the operator assumed. This guide walks through comparing the host mount path, the container path, and the volume type to locate where data is actually being persisted, then verifies with read-only inspection commands.
Open guide →GitHub Actions job is skipped unexpectedly: evaluate condition inputs
Diagnose a GitHub Actions job that is reported as skipped even though it appears in the workflow file. The job is typically excluded by an evaluated `if` condition, a `needs` dependency that was skipped or failed, a matrix entry that did not match, or a workflow-level trigger that does not deliver the expected event payload. The playbook walks through reading GitHub's own UI state for the job, isolating which guard fired, and adjusting only the condition that evidence supports.
Open guide →GitHub Actions fails only on fork pull requests: inspect token permissions
A playbook for diagnosing GitHub Actions runs that succeed on the base repository but fail specifically on pull requests opened from forks. It walks through recognizing GITHUB_TOKEN permission downgrades, separating permission failures from secret-unavailable failures, and choosing an evidence-based remediation path without weakening repository security.
Open guide →GraphQL field is intermittently null: separate resolver errors from missing data
This playbook distinguishes resolver errors from genuinely missing data when a GraphQL field returns null intermittently. It sequences decisions from null propagation rules, to error masking, to dependency-side failure surfaces, and ends with regression checks tied to the GraphQL October 2021 Errors specification.
Open guide →Kubernetes pod stays Pending: locate the scheduling constraint
A pod stuck in Pending means the Kubernetes scheduler cannot bind it to any node. This guide walks engineers from observable symptoms to the specific scheduling constraint blocking placement, using kubectl describe and kubectl get events as the primary evidence sources.
Open guide →Kubernetes service has no endpoints: compare selectors and pod labels
A Kubernetes Service is reachable as a DNS name, but the Endpoints (or EndpointSlice) object behind it is empty. The service selector syntactically parses, yet no Pod labels match it, so the kube-proxy data plane has nothing to program and clients see connection refused or timeouts. This guide frames the failure as a label/selector mismatch and walks through disciplined triage before any edit.
Open guide →Login redirects forever after deploy: compare trusted origins and cookie scope
After a redeploy that changes the public origin (scheme, hostname, or port) of a web application, users hit an infinite login redirect loop on the new host while authentication still works on the old host. The loop is almost always a mismatch between the origins the auth server accepts as return targets and the cookie scope it issued earlier. This guide frames the failure as a boundary problem between identity provider (IdP), reverse proxy, and application origin, and walks through evidence-based comparison of trusted origin allowlists against cookie attributes (Domain, Path, Secure, SameSite) to break the loop without disabling security controls.
Open guide →Logs show success but the user sees failure: reconcile layers and timing
When a success event in logs does not match what the user sees, the fault lies in the boundary between layers or in the way success is defined. This guide walks through reconciling distributed layers and timing so that observability evidence matches user-visible behavior, rather than papering over the gap.
Open guide →Browser heap grows after repeated interactions: compare retained heap snapshots
Browser heap growth after repeated interactions usually points to objects that remain reachable even when the UI no longer references them. The reliable diagnostic is to compare retained heap snapshots taken before and after a scripted interaction sequence, then walk the dominator tree to the GC roots holding the surplus alive. This guide explains the boundaries between UI lifecycle, JavaScript reference graphs, and browser-internal caches that commonly masquerade as leaks.
Open guide →Node.js memory grows from listeners: find the emitter lifecycle mismatch
Guide for diagnosing Node.js heap growth caused by EventEmitter listeners that outlive the request or component that registered them. Explains why setMaxListeners is a tripwire rather than a fix, how to read process.memoryUsage() delta and V8 heap sampling, and how to map retained listeners back to the emitter that created them.
Open guide →Metrics stay flat during an outage: find the instrumented path that disappeared
When an outage is visible to users but your dashboards show flat, unchanging metric lines, the instrumentation has lost visibility of the failing code path. This guide walks engineers through the specific evidence to collect, boundaries to inspect, and conditional fixes to recover the missing telemetry without making assumptions about which side is broken.
Open guide →OAuth works in one browser but not another: compare callback storage
OAuth succeeds in one browser but fails in another because the browser-specific storage of state, nonce, session, and PKCE verifiers diverges between the requestor and the callback. This guide isolates the browser that breaks the round-trip and compares cookie, sessionStorage, localStorage, and service worker behavior against the OAuth provider's requirements.
Open guide →Traces stop at the API boundary: identify missing propagation
This playbook helps engineers diagnose OpenTelemetry traces that terminate at the API boundary instead of continuing into downstream services. The defining signal is that incoming requests produce a root server span, but calls leaving the service (HTTP, gRPC, messaging, DB via instrumented client) appear as new, disconnected root spans in the backend. The guide walks through evidence collection at the boundary, separates propagation from sampling, and scopes fixes to the propagation layer only.
Open guide →PostgreSQL pool queue grows while CPU is low: identify waiting clients
The PostgreSQL connection pool queue grows while server CPU utilization remains low because requests are blocked waiting for a scarce resource the CPU does not measure. Most often these waits are for connections, row-level or transaction locks, or result-set returns from already-running statements. The diagnostic task is to identify which client backends are waiting, on what wait event, and for how long, then correlate that with pool waiters and application latency.
Open guide →PostgreSQL query is fast then slow: find the changing data or plan state
A diagnostic walkthrough for the "fast-then-slow" PostgreSQL pattern: a query that returns in milliseconds on an empty or warm database, then degrades sharply as data volume, cache state, contention, or plan choice shift. The guide organizes the failure into four observable boundaries (data, cache, locks, plan) and shows how to attribute the slowdown to one of them using only read-only views from pg_stat_statements, pg_stat_user_tables, pg_stat_activity, EXPLAIN (ANALYZE), and the statistics collector.
Open guide →PostgreSQL query became slow after deploy: compare plan inputs
When a PostgreSQL query slows down right after a deploy, the most productive first move is to compare the new plan against the old one at the level of inputs the planner actually consumes — statement text, parameter values, table statistics, and index inventory — rather than chasing the symptom in the application.
Open guide →PostgreSQL writes succeed but reads lag: locate the reader target
PostgreSQL writes returning success while subsequent reads return stale or older values usually means the application is reading from a different target than it is writing to. The fix is locating the reader target and aligning it with the writer path before any cache, isolation, or replication tuning is attempted.
Open guide →Queue jobs disappear after acknowledgement timeout: trace ownership
Queue jobs vanish from the visible worker set after an acknowledgement timeout, leaving no clear owner. The playbook traces the handoff from pickup to ack, isolating whether the job was stalled, double-locked, or evicted by a watchdog, and shows how to prove ownership before changing retry logic.
Open guide →Queue latency rises while depth stays flat: find slow consumers or hidden partitions
Queue latency rises while depth stays flat: a beginner playbook for finding slow consumers or hidden partitions, anchored in BullMQ stalled-job mechanics. Use it when age-of-oldest-message climbs even though ready/in-flight counts do not grow.
Open guide →React performance regresses in Chrome: connect profiler evidence to a render path
React performance regressions in Chrome are not single bugs; they are a render path going over budget. This guide connects Chrome DevTools Performance and React DevTools Profiler evidence to a specific commit or component change by enforcing an interaction budget, isolating the render path, and demanding a measurable before/after proof before a fix is accepted.
Open guide →React page slows over time: distinguish retained objects from growing work
Some React pages are not slow at first; they become slow as the user interacts with them. The degrade-over-time symptom is almost always one of two things: retained objects that prevent garbage collection, or work that grows because state, context, or subscriptions keep expanding. Engineers must distinguish between the two before changing code, because the fixes are different and a guess can mask the real cause. This guide frames the decision, then walks through the evidence needed to separate the two classes.
Open guide →Same commit produces different output: identify non-hermetic inputs
A build system returns different artifacts for the same commit hash, indicating a non-hermetic build. This guide walks through identifying uncontrolled inputs (time, environment, network, generated files, dependency resolution) that leak into build outputs, and provides evidence-based steps to prove and remediate the variance.
Open guide →Users receive an old browser bundle after release: trace cache headers and manifest
Users keep receiving an older browser bundle after a new release because an HTTP cache, service worker, or CDN edge is still serving a previously hashed asset set. This guide explains how to trace cache headers and the HTML asset manifest to localize the staleness boundary between origin, CDN, browser memory cache, and disk cache. Use it when post-deploy evidence shows stale JavaScript, CSS, or HTML even though fresh assets exist on origin.
Open guide →Turborepo build is slow despite a cache: identify the uncached task boundary
Turborepo builds that ignore cache are almost always bounded by a single uncached task graph boundary. This guide shows how to identify which task is missing a cache hit, why its inputs/outputs are untracked, and how to prove the fix with a HIT/MISS trace.
Open guide →Turborepo cache differs in CI: compare declared environment inputs
When Turborepo hits locally but misses in CI (or vice versa), the root cause is almost always a difference in declared environment inputs: `env`, `globalEnv`, `globalDependencies`, `dotenv`, or pipeline-level `dependsOn`/`inputs` that diverge between the two environments. The fix is to audit the hash inputs on both sides, align them to the smallest stable set, and then prove parity with a reproducible cache hit.
Open guide →Vite build fails while dev works: compare production-only transforms
Vite builds can fail while `vite dev` succeeds because the production pipeline runs a different code path than the dev server. Dev uses on-demand native ESM with esbuild pre-bundling and runtime transforms, while production uses Rollup with stricter resolution, tree-shaking, minification, and CSS code-splitting. Understanding this asymmetry turns a confusing "works in dev, breaks in build" report into a tractable triage problem with named boundaries.
Open guide →Vite dev server serves stale code: identify the invalidation edge
Vite's dev server can serve stale modules when a change happens outside its tracked invalidation boundary: alias rewrites, dependency pre-bundling cache, config-driven transforms, or filesystem events the watcher misses. This guide frames the failure as a module-graph invalidation edge problem and walks through evidence-driven triage to confirm that a file change actually reaches the client.
Open guide →Container starts and exits immediately: read the process lifecycle
A container that returns to the shell seconds after `docker run` almost always means its PID 1 foreground process terminated. Docker is doing its job; the application inside is not. The lifecycle is observable in `docker ps -a`, in `docker logs`, and in the image's configured Entrypoint and Cmd, so diagnosis starts in metadata before it touches code.
Open guide →Database queries suddenly queue: locate the shared bottleneck
When PostgreSQL throughput collapses without a code deploy, queries typically pile up behind a shared resource: locks held by a long transaction, a saturated connection pool, I/O backpressure on the storage layer, or a plan regression that turns a millisecond scan into a multi-second one. This guide walks database engineers from the first wait_event observation through pg_stat_* evidence to the specific bottleneck, then defines proof criteria before any change is shipped.
Open guide →Deployment succeeds but the application is unreachable
A deployment that finishes without errors can still leave the application unreachable. The build artifact was produced and accepted by the platform, but traffic never reaches the process because of binding, routing, health-probe, or post-deploy startup failures. Diagnose by separating pipeline success from runtime reachability.
Open guide →GraphQL request returns null without an obvious error
GraphQL responses can return null for a field while leaving the errors array empty or empty-looking, leaving engineers with partial data and no obvious failure signal. This guide explains how to distinguish a resolver returning null, an authorization rule masking data, and spec-level null bubbling, then walks through evidence collection, isolation, and verification.
Open guide →Login works once then redirects forever: trace the session contract
Browser appears to authenticate successfully on the first request but is then bounced between the identity provider and the application indefinitely. The root cause is almost always a disagreement in the session contract: cookies that the server believes it set are not the cookies the middleware or callback handler observes on the next hop.
Open guide →CPU is low but requests time out: look beyond compute saturation
CPU utilization is low but client requests still time out, indicating the latency is bounded by something other than compute. This guide frames the symptom as a resource-bound mismatch: time is spent waiting on I/O, locks, connection pools, or downstream deadlines rather than CPU cycles. The first ten minutes are spent measuring wait states and queue depths before considering any code change.
Open guide →