Cache appears updated but users see old data

The symptoms

• •Users report stale data for a key that monitoring shows has been overwritten in the primary cache.
• •Freshness is non-uniform: some users see the new value seconds after the write, while others keep seeing the old value for minutes or until a hard reload.
• •API responses and HTML differ depending on the path taken (e.g., direct API call vs. embedded component fetch), even though both should read the same cache key.
• •Server-side access logs show a cache HIT, but the served payload still matches the pre-update value.
• •Browser DevTools show "from disk cache" or "from memory cache" entries that predate the write, sometimes paired with a strong Cache-Control header.
• •Multiple replicas of the cache cluster appear to disagree about the same key when read from different clients.

Likely causes

• •The reader is connecting to a different cache node or shard than the writer, and topology or consistent hashing has not propagated the update.
• •There is a layered cache (e.g., in-process/heap, local proxy, regional edge, CDN) and only the innermost layer was invalidated.
• •An HTTP/browser cache in front of the application is serving a previously cached response because Cache-Control directives do not force revalidation.
• •TTL or soft-TTL asymmetry means the writer set a long expiry while readers are still serving from a short-lived prior entry.
• •Read-through or cache-aside code reads a stale sibling key (e.g., per-tenant or per-locale namespace) while the writer updated a different key.
• •Negative caching or request collapsing is masking the invalidation by continuing to serve a cached "old" response to simultaneous readers.

First ten minutes

1. 01Confirm the write actually landed: capture the timestamp, key, and value reported by the writer's success log, and the same key/value reported by a direct read against the primary cache endpoint.
2. 02Identify every layer that could answer the read: in-process cache, local proxy, regional cache, edge/CDN, and browser/HTTP cache. Note which layer the writer invalidated and which layer each reader path exercises.
3. 03Reproduce the symptom with cache-bypass by adding a unique query parameter or a hard reload that disables cache, and record whether the new value appears.
4. 04Capture the HTTP response headers from one reproducing user and one fresh client, and compare Cache-Control, Age, ETag/Last-Modified, and any Vary fields.
5. 05Read the same key from at least two different cache nodes (by routing or by client) and compare values to detect replica divergence.

Evidence to collect

• •Writer-side evidence: timestamp of the successful write, key, value hash or version, and the cache node or shard the write targeted.
• •Reader-side evidence: for at least two users, the cache key, cache node, HIT/MISS status, age, and the served payload bytes or hash.
• •HTTP evidence: full Cache-Control, Age, ETag, Last-Modified, Vary, and Date headers from the reproducing request and a freshly issued request.
• •Topology evidence: hash ring, shard map, or routing table that shows which client or region maps to which cache node for the affected key.
• •Layer inventory: list of every cache layer that can answer the read, with TTL settings, invalidation hooks, and whether each hook ran for this update.

Where to look

• •The application boundary between the writer and the reader: serialization, key derivation, and namespace prefixing.
• •The cache client boundary: connection pool, consistent-hash ring, replica set selection, and any read-from-replica or read-from-primary policy.
• •The HTTP cache boundary described by the MDN Cache-Control reference: directives such as max-age, s-maxage, no-cache, no-store, and private/public.
• •The browser boundary: memory cache, disk cache, service worker cache, and any fetch handler that intercepts requests.
• •Edge and CDN boundaries: regional POPs, purge queues, and surrogate keys that govern which response keys are invalidated.

Diagnostic steps

1. 01Step 1: Verify the write landed by reading the same key directly from the cache using the writer's connection and an independent connection; compare value and version.
2. 02Step 2: Identify the reader's cache node by inspecting the cache client logs or metric labels (node id, shard, region) for the reproducing request.
3. 03Step 3: Compare the reader's cache node to the writer's cache node; if they differ, the suspect is topology/propagation rather than invalidation logic.
4. 04Step 4: Bypass every application-level cache layer in order (innermost to outermost) and re-test; the first layer whose bypass yields the new value is the stale layer.
5. 05Step 5: Inspect HTTP responses per the MDN Cache-Control directives; if Age is present and positive, or if max-age/s-maxage is large, the HTTP/shared cache is the suspect.
6. 06Step 6: Inspect the browser for service worker registrations and for disk/memory cache entries; if the response is served from the HTTP cache, the browser layer is the suspect.
7. 07Step 7: Compare key derivation between writer and reader (tenant, locale, version, hash of input) to detect a sibling-key mismatch.

Common mistakes

• •Concluding the write is broken when the write is correct but the reader is routed to a different replica or layer.
• •Invalidating only the primary cache while a regional cache, CDN, or in-process cache continues to serve the prior response.
• •Setting a short TTL on the writer but reading through a long-lived HTTP cache that ignores application-level invalidation.
• •Trusting a single cache HIT log line as evidence of freshness without checking the value or the Age.
• •Hashing keys differently on writer and reader (e.g., different tenant prefix, locale, or sharding scheme), so the read misses the written entry.
• •Ignoring the browser cache because the API is "internal," while the application or service worker is still caching the response.

Safe fixes

• •If the writer and reader hit different cache nodes, change the reader to read from the primary or from the node that holds the freshest write, and verify consistency before scaling back.
• •If a sibling HTTP cache is the stale layer, attach Cache-Control directives that force revalidation (no-cache) or that disable caching (no-store) for the affected route, per the MDN Cache-Control reference.
• •If a layered cache is the stale layer, invalidate every layer in the read path, ordered from outermost to innermost, and confirm each invalidation hook returned success.
• •If key derivation is inconsistent, standardize the key construction and add a correlation log that prints the canonical key on both write and read paths.
• •If the browser cache is the stale layer, append a content-version or build-id to the URL so previous cached entries are not reused, and verify in DevTools that the response is no longer served from disk or memory.
• •Apply each fix only after the matching diagnostic step has identified that layer as the stale one; do not blanket-disable layers.

Prove the fix

1. 01Two independent clients reading the same key from different cache nodes return the post-update value, and HIT logs show the new value's hash or version.
2. 02HTTP responses for the affected route show Age values that reset to zero on each successful update, and ETag/Last-Modified reflect the new content.
3. 03An automated regression check that writes a sentinel value, then reads it from every cache node and every layer in the read path, succeeds with no stale hits.
4. 04Browser DevTools no longer show prior responses served from disk or memory cache for the affected URL after the update completes.
5. 05Synthetic monitoring continues to record the new value across regions and across at least two consecutive read cycles.

Prevention and next steps

• •Maintain a single, documented inventory of every cache layer that can answer a read, with the invalidation hook or purge command required for each.
• •Standardize key derivation in a shared library and log the canonical key on both write and read paths so mismatches are visible immediately.
• •Define HTTP cache directives per route and review them when routes change, anchored to the MDN Cache-Control reference.
• •Run a periodic end-to-end freshness check that writes a sentinel value and reads it from every cache node and every layer; alert on any stale hit.
• •Track replica selection and read-from-replica policy in the cache client configuration so readers can be steered away from lagging replicas without code changes.

Safe commands and checks

redis-cli -h <host> -p <port> GET <key>
redis-cli -h <host> -p <port> OBJECT IDLETIME <key>
redis-cli -h <host> -p <port> CLUSTER NODES
redis-cli -h <host> -p <port> CLUSTER KEYSLOT <key>
redis-cli -h <host> -p <port> INFO replication
curl -sI -H 'Cache-Control: no-cache' <reproducing-url>
curl -sI -H 'Pragma: no-cache' <reproducing-url>
curl -sI -H 'If-None-Match: <etag>' <reproducing-url>