04 / Verification
A fix is only real when you can prove it holds.
Regression and verification guides for recovery, idempotency, backoff, invalidation, migrations, timeouts, and rollback.
How to verify an API client honors 429 backoff
A verification playbook for confirming whether an HTTP API client honors HTTP 429 (Too Many Requests) backoff guidance, including the Retry-After header and a defined retry budget. The guide treats 429 as a throttling signal that demands patience, not persistence, and frames verification around three observable behaviors: honoring the delay, capping retries, and recording evidence that downstream exhaustion is bounded rather than amplified.
Open guide →How to verify an API recovers after an upstream reset
An advanced, evidence-driven walkthrough for confirming that an HTTP API successfully recovers after an upstream connection reset without duplicating side effects. The guide focuses on reset-classification behavior at the client or gateway boundary, observable signal patterns, and a deterministic verification protocol using safe, read-only checks anchored to the HTTP 502 specification.
Open guide →How to test an API idempotency key under duplicate delivery
Idempotency keys are designed so that a single logical key submitted across duplicate HTTP deliveries produces exactly one durable side effect and one canonical response. This guide walks engineers through a deterministic test plan for verifying that property, the evidence required to distinguish a true idempotency contract failure from a transport-level symptom such as a 502 from an upstream gateway, and the checks that prove a fix holds under re-delivery. The emphasis is on observable behavior at the API boundary: response equality, storage of the key, and refusal of conflicting bodies with the same key.
Open guide →How to verify API timeout cancellation releases resources
When an HTTP API request times out, the receiving system must demonstrably cancel in-flight work and return any client handle, socket, or stream it was holding. This guide defines what engineers should observe, measure, and assert to verify that timeout-driven cancellation actually releases resources rather than merely aborting the caller. The argument: timeout behavior is a contract about resource ownership, and a 502-class failure or a hung client is often evidence that the contract is being violated silently.
Open guide →How to prove browser event listeners are removed
Proving that browser event listeners are actually removed after unmount requires observable evidence, not assumptions. This verification guide shows how to detect retained listeners from repeated mount/unmount cycles using framework-agnostic tooling and the browser Performance APIs.
Open guide →How to verify a browser long-task performance fix
This guide explains how to verify that a browser long-task performance fix has actually removed an interaction from waiting behind a long main-thread task. It focuses on measurement before and after the change, using browser Performance APIs and the Long Tasks API, so engineers can confirm an observable improvement rather than rely on assumed gains.
Open guide →How to verify a browser memory regression against a baseline
This guide shows engineers how to verify a browser memory regression against a captured baseline. It focuses on the failure mode where repeated workflows in a web application retain an unbounded heap instead of converging to a steady state, and walks through disciplined baseline capture, controlled replay, and statistical comparison.
Open guide →How to verify session cookie behavior across the login redirect
This guide explains how to verify that a session cookie is actually attached to the redirect callback and the subsequent API boundary after login. The failure mode is precise: the browser must send the session cookie at the intended callback and API boundaries, and SameSite classification quietly determines whether that delivery happens at all. We treat verification, not configuration, as the deliverable.
Open guide →How to test CSRF token rotation without invalidating active forms unexpectedly
CSRF token rotation is meant to refresh the anti-forgery credential without abandoning tokens still bound to in-flight form submissions. The bug to verify is the opposite: rotation that silently invalidates active forms, or rotation that preserves stale tokens beyond their intended window. This guide frames rotation as a state-transition contract — pre-rotation tokens, the rotation event, and post-rotation tokens — and gives engineers a way to test the boundaries of that contract from observable HTTP evidence alone. The contract has three observable parts: the Set-Cookie attributes that carry the new token, the server-side acceptance window applied to submitted tokens, and the synchronization rules that decide which form-render moment triggers rotation.
Open guide →How to verify a deployment rollback with compatible state
Engineers face a subtle verification problem after a deployment rollback: the previous release is now serving traffic, but it must be proven to interoperate with the current schema, configuration, and downstream contracts. This guide frames rollback verification as a compatibility check between the rolled-back binary and the live state, not as a restart confirmation. It covers observable signals, decision boundaries, and proof criteria for a rollback that is correct on paper but unsafe in practice.
Open guide →How to verify Docker port publication reaches the intended process
Verifying Docker port publication is a binding-and-route diagnostic, not a connectivity check. The goal is to confirm that a `-p`/`--publish` flag on `docker run` translates into a host port whose socket terminates inside the containerized process bound to the intended container port, without accidentally publishing on a routable interface or to a wrong listener.
Open guide →How to verify Docker volume persistence across replacement
Verification playbook for confirming that data written to a Docker volume survives container removal and replacement. It distinguishes named volumes, anonymous volumes, and bind mounts using only the docker run reference, since that is the supplied source anchor. The guide frames persistence as an evidence-driven claim, not an assumption, and shows how to prove survival with paired writes, reads, and diffs against the host path.
Open guide →How to verify recovery after a database commit and external side effect diverge
Guide to verifying recovery after a distributed transaction commits locally but diverges from an external side effect (queue, cache, search index, webhook, third-party API). It defines a divergence contract, an evidence checklist, and a reconciliation verification path that proves the split state is observable and repairable without destructive operations.
Open guide →How to verify GitHub Actions artifact lineage
Verify that a GitHub Actions artifact downloaded by a consumer was produced by the intended workflow run, ref, and commit before trusting its contents. This playbook walks through collecting run metadata, comparing it to local expectations, and confirming artifact identity using read-only GitHub CLI commands anchored to the Actions workflow syntax documentation.
Open guide →How to verify GitHub Actions permissions are least-privilege
Tightening GitHub Actions permission scopes by reading the workflow's actual API surface, comparing it against declared permissions, and walking the change through branch protection, environments, and a workflow_dispatch smoke test before merge.
Open guide →How to verify graceful shutdown under in-flight requests
Verification guide for confirming that a service performs graceful shutdown under in-flight requests. Walks through observable signals, an ordered ten-minute triage, boundary-specific inspection points, diagnostic steps that separate SIGTERM misconfiguration from listener and hook defects, conditional safe fixes, and a regression check that re-proves the policy under load.
Open guide →How to verify GraphQL errors preserve field paths
A verification playbook for confirming that a GraphQL service still surfaces `errors` with `path` and `extensions` arrays attributable to the specific field and input that caused each partial failure, rather than flattening failures into a top-level "request failed" message that loses field-level attribution. Focuses on the conformance behavior defined in the GraphQL October 2021 specification, Section 6.4.3, and on reproducible inspection of resolver, transport, and framework boundaries.
Open guide →How to test JWT expiry around clock disagreement
JWT verification around the expiry boundary is governed by explicit clock tolerance policy, not by raw timestamps. This guide frames the verification task: decide and prove whether tokens landing on or just past exp, nbf, or iat should be accepted, rejected, or queued for refresh, given documented skew between issuer and verifier clocks.
Open guide →How to test Kubernetes memory behavior before OOMKilled
Memory pressure in Kubernetes is observable and testable before the kernel terminates a workload. This guide shows how to design a verification run that exercises the cgroup memory boundary, distinguishes OOM from throttling, and produces reproducible evidence of behavior under pressure.
Open guide →How to verify Kubernetes readiness removes and restores traffic correctly
Kubernetes readiness probes are evaluated by the kubelet and surfaced as the Ready condition on the Pod. When a probe fails, the Pod's IP is removed from the matching Endpoints (and EndpointSlice) object, and kube-proxy stops routing cluster traffic to it. The contract to verify is bidirectional: failing pods must stop receiving traffic, and once they recover they must become eligible again. Without an explicit test, a misconfigured probe can keep a "NotReady" pod in the Service load-balancing pool, or it can permanently block recovery after a transient failure.
Open guide →How to verify lock lease renewal across a pause
This playbook walks a beginner through verifying that a distributed lock lease actually renews across a pause (GC, stop-the-world, network blip, process suspension) instead of silently expiring and being reissued to another owner. The verification contract is that ownership must persist only as long as the owner can still prove liveness — a paused process that cannot send heartbeats must lose the lease, and a resumed process must re-prove liveness before it is trusted again.
Open guide →How to verify log sampling preserves rare failure evidence
Learn how to verify that a sampled log pipeline still retains the correlated events needed to reconstruct a rare production failure. This guide provides a deterministic evidence-collection workflow for engineers auditing tail-based or head-based sampling, using observable properties of trace and log correlation rather than vendor-specific dashboards.
Open guide →How to test lost-update protection with concurrent writers
A practical playbook for verifying lost-update protection in concurrent writer scenarios, focused on revision, version, or row-level guards. The guide frames the verification task around observable boundaries: optimistic concurrency exceptions, row-version mismatches, merge-or-reject outcomes, and the difference between application-layer guards and database transaction isolation.
Open guide →How to verify OAuth state is bound to the initiating browser session
OAuth's `state` parameter is a per-flow nonce that must be bound to the browser session that initiated the flow; this guide explains how to verify that binding before accepting any authorization callback, using cookie-scoped storage and explicit comparison.
Open guide →How to verify OpenTelemetry context crosses async work
Verify that OpenTelemetry context (trace_id, span_id, traceparent/tracestate) survives when work crosses an async boundary such as a queued task, callback, thread-pool handoff, or event-driven continuation. The guide covers what to inspect, which boundary to capture, how to read a captured snapshot, and how to confirm a child span is still connected to the trace that started the request.
Open guide →How to verify a PostgreSQL CTE plan after a performance fix
Engineers fixing a slow PostgreSQL CTE must verify the revised plan still matches the intended scan and join behavior, since PostgreSQL can rewrite CTEs (inline or as initPlans) and change node types without warning. This guide walks through confirming the plan before, during, and after a performance fix using EXPLAIN, pg_stat_statements, and the documented statistics views.
Open guide →How to verify a PostgreSQL lock waiter recovers
Engineers often see a session blocked on a row or relation lock and need to confirm that the waiter will resume (or fail predictably) once the holder commits, rolls back, or is terminated. This guide frames that as a verification task: prove recovery, do not assume it. It walks through observing the wait in pg_stat_activity and pg_locks, identifying the blocker, and confirming the waiter advances only after the holder releases the lock.
Open guide →How to prove PostgreSQL pool clients are released on errors
Engineers suspecting connection leaks in PostgreSQL pools need explicit evidence that every code path returns the borrowed client. This guide frames the verification task: instrument both success and failure branches, then prove release against PostgreSQL's own statistics views rather than pool-side counters alone.
Open guide →How to test PostgreSQL statement timeout without hiding locks
This guide explains how to verify PostgreSQL statement_timeout behavior in isolation from lock-wait latency, so engineers can prove that a deadline fired against an intended statement rather than being masked by a concurrent lock. It frames verification as a controlled experiment with three observable boundaries: timer enforcement, error surface, and preserved evidence in pg_stat_activity and pg_stat_statements.
Open guide →How to verify PostgreSQL serialization failures retry safely
Verification guide for confirming that a PostgreSQL application properly retries only the repeatable work after a serialization failure (SQLSTATE 40001), rather than re-running side effects, dropping transactions, or masking other errors as retryable. The guide targets engineers who have observed 40001 errors in logs or monitoring and must prove that the retry boundary is correct, bounded, and safe under concurrent load.
Open guide →How to verify poison messages are isolated
Playbook for verifying that a poison message in a BullMQ-based queue is correctly isolated so it cannot consume normal worker capacity. Walks through triage of stalled-job signals, queue-side evidence collection, decision points on retry vs. delayed vs. failed state, and a regression check that proves the bad message is parked without starving healthy jobs.
Open guide →How to verify queue visibility extension for slow jobs
Playbook for verifying that a Queues worker is correctly extending visibility (lock) for slow jobs, so that another instance cannot redeliver the same job mid-processing. Defines when extension is mandatory, what evidence proves it is happening, and how to detect duplicate work caused by stalled-job recovery.
Open guide →How to verify React effect cleanup after unmount
Verifying that React useEffect cleanup actually fires and fully releases subscriptions, timers, and in-flight requests after a component unmounts. The guide treats unmount ownership as a verifiable contract, not a hope, and frames React's effect lifecycle as the boundary the developer must prove holds in production.
Open guide →How to verify a React render performance fix
A verification protocol for confirming that a React render performance fix actually reduces render work while preserving correctness updates. Covers baseline capture with the Profiler, controlled before/after comparisons, isolation of memoization, keying, and effect-deferral changes, and the regression checks that distinguish a real render-budget win from a misleadingly quiet component tree.
Open guide →How to verify a build is reproducible from the same commit
Reproducible builds mean that starting from the same declared inputs—commit, source tree, toolchain, and environment—two independent clean builds produce byte-identical (or hash-equivalent) outputs. This guide shows how to engineer the verification step itself: capture the inputs, isolate them, build twice, and compare. The failure mode being diagnosed is non-determinism that appears only across environments or reruns, not a single broken artifact.
Open guide →How to verify a Turborepo cache hit is safe
Verification playbook for determining whether a Turborepo "cache hit" is genuinely safe — meaning the restored outputs reproduce the same artifacts only when all relevant inputs (sources, dependencies, environment, configuration) match what produced them originally.
Open guide →How to verify Turborepo cache inputs include environment dependencies
Turborepo caches task outputs based on a content hash derived from declared inputs. When an environment dependency changes but is not listed in a task's `inputs`, the hash is unchanged, the cache returns a stale artifact, and the task fails to invalidate. This guide shows how to verify that environment variables, dotenv files, and related inputs are wired into a Turborepo task so that the cache invalidates correctly.
Open guide →How to test Vite alias resolution in dev and production builds
Verify that Vite alias configuration resolves identically in dev (esbuild pre-bundling + on-demand transform) and production (Rollup bundling) output. The guide frames alias resolution as a contract: every import the resolver accepts in dev must survive bundling without rewriting, dead-code elimination, or externalization.
Open guide →How to verify Vite dependency invalidation after an upgrade
How to verify that Vite's dependency pre-bundle graph stops serving transformed output from the previous version after an upgrade, including cold-cache checks, metadata comparison, and an HMR-based regression test.
Open guide →How to prove workers acknowledge after durable commit
Prove that queue workers only acknowledge after a durable commit so that a crash before commit leaves work recoverable rather than falsely completed. This playbook focuses on the worker-to-broker acknowledgement boundary, not generic queue tuning, and uses stalled-job semantics as the evidence anchor for the contract.
Open guide →How to load-test a database connection pool
Run a controlled load test that proves a database connection pool stays bounded: acquired count never exceeds configured max, pending queue never grows without bound, and no connection survives past its expected lifetime after load ends. Treat the test as a contract, not a benchmark—define ceilings for acquired, pending, and p99 latency before ramping, and verify each ceiling after.
Open guide →How to prove a lock releases after a worker crash
Regression-test recipe for proving that a distributed lock's ownership safely transfers to a new holder after the original lock owner crashes or disappears. Focuses on observable evidence (lease expiry, fencing monotonicity, refusal of stale owner writes) rather than assuming the lock simply "times out". Maps failure boundaries to verifiable assertions suitable for CI.
Open guide →How to reproduce and verify cache invalidation
How to reproduce and verify cache invalidation: a disciplined write-then-read protocol that proves a single write removes or replaces every reader-visible representation of a key, across the cache layer, application layer, and downstream stores. The guide treats invalidation as a testable contract, not a guess.
Open guide →How to test a database migration rollback safely
A verification guide for proving that a database migration and its rollback leave a recoverable schema and data state. It walks engineers from observable pre-migration evidence through controlled rollback execution to a defensible post-rollback proof, with a focus on detecting schema drift, lost rows, and non-reversible DDL before they reach production.
Open guide →How to test idempotent webhook handling
A verification workflow for confirming that a webhook consumer accepts at-least-once delivery without producing duplicate side effects. The guide frames idempotency as a contract to test against: replay a known event_id, observe the dedupe store, and assert that downstream row, queue, and notification counts remain at one. It distinguishes payload-based dedupe from provider-id-based dedupe and treats signature verification as a precondition that is never bypassed during testing.
Open guide →How to validate graceful shutdown under load
Guide to validating that a long-running service stops accepting new work, drains or rejects in-flight work predictably, and exits within a bounded window when signaled under load. Covers signal handling, readiness flip, connection draining, and post-shutdown health checks as a reproducible verification task.
Open guide →How to verify retry backoff instead of retry storms
Engineers need a way to prove that a retry policy is actually spreading work and stopping at a bounded budget, not just trusting that "exponential backoff with jitter" is configured. This guide defines a verification procedure: measure inter-attempt gaps, confirm jitter width, confirm an upper bound on attempts, and confirm downstream load is uncorrelated with the original failure.
Open guide →How to verify timeout behavior end to end
Verifying timeout behavior end to end means proving that a request stops waiting at the intended deadline, returns a deterministic error, and releases its resources. This guide gives engineers a disciplined verification workflow: define the deadline boundary, observe the actual abort point, confirm the error shape, and prove no late completion or resource leak follows.
Open guide →