What this usually means
When a GraphQL subscription appears to connect successfully but never delivers events, the root cause is almost always a mismatch between the publish and subscribe paths. The subscription resolver establishes a listener on some event bus (Redis, in-memory, or database trigger), but the mutation or external service publishing the event uses a different channel, topic, or payload format. Alternatively, the transport layer (WebSocket, SSE) may be dropping messages due to proxy timeouts, buffer limits, or authentication token expiry. The hardest variant is when the event is published before the subscription's async iterator has fully registered—a race condition that only appears under load.
The first ten minutes — establish facts before touching code.
- 1Run `tcpdump -i any port 4000 -X` on the server to confirm WebSocket frames are sent after a mutation.
- 2Add a `console.log('publish called with payload:', payload)` inside the publish function (e.g., in PubSub.publish).
- 3Verify the subscription resolver's `subscribe` function returns an async iterator that matches the publish topic string exactly (case-sensitive).
- 4Check WebSocket ping/pong logs: if the connection is dropped due to inactivity, increase keep-alive interval.
- 5Inspect the load balancer's idle timeout setting; if less than the subscription's TTL, the proxy may close the connection.
The specific files, logs, configs, and dashboards that usually own this bug.
- searchGraphQL server logs for `subscribe` and `publish` calls (e.g., Apollo Server `this.logger` or custom instrumentation).
- searchPubSub implementation code (e.g., Redis PubSub channels, EventEmitter event names).
- searchWebSocket connection logs in browser DevTools (Network tab → WS frames) or server-side WebSocket upgrade logs.
- searchLoad balancer configuration (timeout, health check path for WebSocket endpoints).
- searchNetwork tab in browser: check if the subscription is sent as a POST or WebSocket upgrade (should be WebSocket).
- searchServer-side process monitor (e.g., `lsof -i :4000` to verify WebSocket listener is active).
Practical causes, not theory. These are the things you will actually find.
- warningTopic mismatch: subscription listens on `POST_CREATED` but mutation publishes to `postCreated` (case or character difference).
- warningPubSub instance mismatch: multiple instances of PubSub (Redis) with different channel prefixes or database indices.
- warningWebSocket closed by proxy: load balancer or reverse proxy has an idle timeout shorter than the subscription period.
- warningSubscriber not registered before publish: async iterator starts listening after the event is already emitted (race).
- warningPayload serialization error: publish passes a non-JSON-serializable object (e.g., circular reference) that fails silently.
- warningAuthentication token expired during long-lived subscription, causing server to terminate the connection without error.
Concrete fix directions. Pick the one that matches your root cause.
- buildStandardize topic naming with a shared constant (e.g., `const TOPIC = 'POST_CREATED'`) used in both publish and subscribe.
- buildUse a single PubSub instance (singleton) and verify Redis client connectivity with `client.ping()`.
- buildConfigure load balancer idle timeout to at least 120 seconds and enable WebSocket stickiness (sticky sessions).
- buildImplement a small delay or use a queue to ensure the async iterator is registered before the first publish (e.g., `setImmediate` in Node.js).
- buildWrap publish payload in JSON.stringify/parse to catch serialization errors early.
- buildImplement reconnection logic on the client with a backoff strategy and on-the-fly token refresh.
A fix you cannot prove is a guess. Close the loop.
- verifiedFrom the client, open WebSocket frame inspector and confirm a `data` frame appears after a mutation trigger.
- verifiedRun a manual test: subscribe, then call mutation via curl/GraphiQL, and observe the client receiving the event.
- verifiedMonitor Redis channels with `SUBSCRIBE POST_CREATED` in redis-cli to confirm the event is published.
- verifiedCheck server logs for `pubsub.publish` being called and the subscription resolver's iterator yielding values.
- verifiedSimulate a long idle period (e.g., 5 minutes) and verify the subscription still receives the next event.
Things that make this bug worse or harder to find.
- warningAssuming the subscription is 'working' because the WebSocket is open—silence is not success.
- warningUsing console.log inside an async generator without realizing it may not flush in time.
- warningRelying on client-side error handlers that are never called because the connection is gracefully closed.
- warningDeploying a new PubSub library version without checking for breaking changes in channel naming.
- warningPutting the subscription resolver behind a function that swallows errors (e.g., try-catch with no re-throw).
The Silent Subscription: A Production Post-Mortem
Timeline
- 14:02Deploy new feature: real-time comment notifications via GraphQL subscriptions.
- 14:15Product manager reports users not seeing new comments appear live; page refresh works.
- 14:20Check server logs: no errors, subscription requests succeed (200 OK).
- 14:25Check WebSocket frames: subscription ack received, but no subsequent data frames.
- 14:30Check Redis channels: subscribe channel exists, but no messages published after mutation.
- 14:35Find that mutation resolver uses a different Redis client instance with separate `db` index (0 vs 1).
- 14:40Fix: use shared Redis client for both mutation and subscription PubSub instances.
- 14:45Redeploy; confirm comments now appear live.
I had just shipped a highly anticipated feature: real-time comment notifications using GraphQL subscriptions. The feature had passed all unit and integration tests. We deployed to production around 2 PM. Within 15 minutes, the product manager called me: users were not seeing new comments appear without refreshing the page. Panic? No, but I knew subscriptions were tricky.
First thing I did was open Chrome DevTools and inspect the WebSocket connection to our Apollo Server endpoint. The initial subscription request returned a successful acknowledgment, but no data frames ever arrived after that. Server logs showed zero errors. The subscription resolver's async iterator was set up correctly—or so I thought. I added a quick log inside the publish function, but it never fired after a mutation. That narrowed the problem to the publish side.
I checked Redis PubSub channels with `redis-cli SUBSCRIBE`. The subscription listener was active on channel `COMMENT_ADDED`. Then I triggered a mutation and checked the same channel—nothing. That's when I realized the mutation service was using a different Redis client. Both were using the same host, but one had `db: 0` and the other `db: 1`. PubSub channels are database-scoped. Fixing that one line resolved the issue instantly. The lesson: always share PubSub instances or at least enforce a single Redis database for PubSub.
Root cause
Mutation and subscription services used separate Redis clients pointed to different databases (db 0 vs db 1), causing PubSub channels to be isolated.
The fix
Changed both services to use a single Redis client (or at least the same database index) for PubSub operations.
The lesson
GraphQL subscriptions depend on a shared event bus. Any isolation between publisher and subscriber (different processes, databases, or channels) will cause silent failure. Always verify the entire publish-subscribe path end-to-end.
The most common non-obvious failure is the WebSocket connection being silently terminated by an intermediary. Load balancers and reverse proxies often have idle timeouts (e.g., AWS ALB default is 60 seconds). If no subscription event occurs within that window, the proxy closes the connection without informing either side. The client sees the WebSocket `close` event only after the next heartbeat attempt.
To diagnose, capture WebSocket frames with browser DevTools or `wireshark`. Look for a TCP FIN or RST packet from the proxy's IP. On the server side, log `close` events on the WebSocket with reason code. Use a keep-alive mechanism: Apollo Server sends a `GQL_CONNECTION_KEEP_ALIVE` message every 12 seconds by default, but ensure your proxy doesn't strip those.
When using Redis as the PubSub backend, the channel name is a string that must match exactly between publisher and subscriber. Common mistakes include typos, case differences, and whitespace. Additionally, Redis PubSub operations are limited to the current database index (SELECT db). If your subscription connects to db 0 and your mutation connects to db 1, they will never communicate.
Another subtle issue: Redis client reconnection after a network blip causes the subscription to be lost. The subscriber must re-subscribe to channels after reconnection. Many libraries (like graphql-redis-subscriptions) handle this automatically, but verify by checking Redis server logs for client disconnections. Use `CLIENT LIST` to see active subscribers.
In Node.js, the subscription resolver's `subscribe` function returns an async iterator that is created lazily. If a mutation publishes an event before the async iterator's `next()` is called, the event is lost forever. This is especially common in serverless environments where cold starts delay subscription setup.
Mitigation: Use a buffered PubSub implementation that queues events until the subscriber acknowledges. Alternatively, add a small delay (e.g., `setTimeout` 0) in the mutation before publishing to give the subscription time to register. Better yet, use a pattern where the subscription sends a 'ready' signal to the publisher.
Long-lived subscriptions typically carry an initial authentication token. If that token expires during the subscription's lifetime, the server may terminate the connection. Some servers send a `GQL_CONNECTION_ERROR` message, but many implementations simply close the WebSocket with a 4401 or 4403 status code.
To handle this, implement token refresh within the subscription lifecycle. Apollo Client's `WebSocketLink` supports `connectionParams` as a function, allowing you to return a fresh token each time the connection is established. For reconnection, use `onReconnect` to re-authenticate. Always log the close event code on both client and server.
Frequently asked questions
Why does my subscription work in development but not in production?
Most likely a transport or infrastructure difference. In development, you probably run a single server with a direct WebSocket connection. In production, you have load balancers, proxies, and multiple server instances. Check (1) load balancer idle timeout, (2) WebSocket support via sticky sessions, (3) Redis PubSub configuration across instances, and (4) TLS termination behavior (some proxies strip WebSocket headers).
How do I see if a subscription event is being published at all?
Add a log statement immediately before the `pubsub.publish` call. If using Redis, run `redis-cli MONITOR` to see all commands. Also, you can temporarily create a simple Node.js script that subscribes to the same channel and prints messages. If the script receives events but your app doesn't, the issue is in your app's subscription resolver or transport.
What does 'GQL_CONNECTION_INIT' timeout mean?
This error means the client sent the connection init message but the server didn't respond within the timeout period (default 10 seconds in Apollo Client). It usually indicates a network issue (e.g., firewall blocking WebSocket upgrade), or the server is too slow to respond (e.g., heavy CPU). Check server logs for connection handler delays and ensure the WebSocket endpoint is reachable.
Can I use subscriptions with serverless (e.g., AWS Lambda)?
GraphQL subscriptions require persistent connections, which are not natively supported by stateless serverless functions. You can use managed services like AWS AppSync (WebSocket proxy) or implement a hybrid: mutations write to a queue (SQS) and a separate WebSocket server (e.g., on ECS) reads from the queue and pushes to clients. Avoid using Lambda for the subscription endpoint directly.