Deployment succeeds but the application is unreachable

The symptoms

• •Deployment pipeline reports success, yet HTTP requests to the service URL time out or return connection refused when probed externally.
• •Load balancer, ingress, or service-mesh health checks remain in a failing or unknown state after the rollout completes.
• •The platform dashboard shows the new revision as healthy while an independent external monitor reports a 100 percent error rate.
• •The container or process appears running in the orchestrator but no listener is bound to the expected port on any non-loopback address.
• •Application logs are empty or stop abruptly at startup, with no crash record because the process is still alive but never reached its listen call.
• •DNS resolves the service hostname to a concrete address, but a TCP probe to the expected port on that address receives no response.

Likely causes

• •The process binds only to a loopback address instead of all interfaces, so external traffic cannot reach it; per the 12factor.net port-binding guidance, the port should be declared and exposed rather than hardcoded to a private address.
• •A health or readiness probe returns a non-success status, causing the load balancer or orchestrator to remove the instance from rotation even though the build itself succeeded.
• •A post-deploy migration, secret lookup, or configuration step throws and silently leaves the process in a half-started state with no listening socket.
• •Ingress, service-mesh, or load-balancer rules still reference an old selector, port, or hostname that the new revision does not satisfy.
• •Network policies, security groups, or firewall rules block traffic to the port, or block the path between the ingress and the backing instance.
• •The container image starts a different entrypoint than expected, so the listening server never starts even though the container itself is reported as running.

First ten minutes

1. 01Confirm whether the symptom is connectivity versus application errors by running an in-region probe from a known-good host, not the public internet.
2. 02Check the orchestrator view of the process: is the pod or task in Running state, and does it report a bound port matching the configured containerPort or targetPort.
3. 03Inspect the most recent process logs for the time window around the rollout completion; look for startup exceptions, binding errors, and the moment a listener is supposed to open.
4. 04Look at the load balancer or ingress target health and the corresponding probe configuration; verify probe path, port, scheme, and expected status code.
5. 05Resolve the public hostname to a concrete address and confirm that address matches the expected backing instance or service IP after the deployment.
6. 06Avoid rolling back immediately; capture the evidence listed here first, because a rollback will erase the in-memory state needed to confirm the cause.

Evidence to collect

• •Timestamped process or container logs covering at least the rollout completion time plus one full readiness window before and after.
• •Output from a local socket query on the running host, showing which addresses and ports the application process has bound.
• •Ingress, load balancer, or service-mesh target-health entries with the failure reason recorded by the platform, not just the status.
• •The effective configuration applied to the new revision: environment variables, mounted config, command, and arguments, compared with the previous known-good revision.
• •Network reachability data such as a traceroute from the ingress to the backing instance, plus any network policy or security-group rules that apply.

Where to look

• •The container or process startup logs filtered by the deploy timestamp, especially the first thirty seconds after process start.
• •The platform's target-group, service, or endpoint object that maps the external hostname to backing instances, and its selector versus pod labels.
• •The health-check definition in the ingress, load balancer, or orchestrator probe, including path, port, scheme, and success criteria.
• •The application listen configuration: whether it reads a PORT or BIND_ADDRESS environment variable and what defaults it falls back to.
• •DNS records for the public hostname, especially during a rollout window where records may briefly point to old and new backends.

Diagnostic steps

1. 01From inside the cluster or on the host, attempt a direct TCP connection to the application's expected port; success here with failure from outside suggests a routing or firewall issue rather than a process issue.
2. 02Read the application startup log and confirm a line that states the process is listening on a specific address and port; absence of this line indicates the listener never opened.
3. 03Query the orchestrator for the readiness or health-probe status of each instance backing the deployment; an instance that is Running but not Ready is the classic signature of a failed probe.
4. 04Compare the new revision's effective configuration with the previous known-good revision using a diff; even small differences in bind address or port can break reachability.
5. 05From a host that shares the network path as the load balancer or ingress, issue the same probe the load balancer would issue and observe the status code returned by the application directly.
6. 06Check whether the platform recorded any post-deploy hooks, migration steps, or init container failures that completed successfully from the platform's perspective but blocked the application from opening its listener.
7. 07Capture a packet trace on the application host during an external probe to see whether SYN packets arrive at all, which distinguishes routing failure from application-level refusal.

Common mistakes

• •Treating a green pipeline run as proof of reachability; the pipeline only confirms that an artifact was produced and accepted, not that it is serving traffic.
• •Restarting the application as the first response, which destroys the in-memory state and logs needed to identify why the listener never opened.
• •Looking only at error logs when the symptom is no logs at all; a process that never reaches its listen call produces silence, not exceptions.
• •Changing probe thresholds or disabling health checks to force traffic to flow, which masks the underlying reachability problem and produces a service that returns errors rather than refusing connections.
• •Assuming the previous revision is healthy because the platform dashboard says so; verify with an external probe, since dashboards often reflect the desired state rather than observed behavior.

Safe fixes

• •Configure the process to bind to the address required by the platform, typically all interfaces, 0.0.0.0, or the value of an injected BIND_ADDRESS variable, per the 12factor.net port-binding guidance that ports should be declared and exposed rather than hardcoded.
• •Align the readiness probe's path, port, and expected status code with the actual endpoint exposed by the application, and confirm that the probe runs from the same network namespace the platform expects.
• •After any configuration change, redeploy and re-check both the orchestrator readiness state and an external HTTP probe before declaring the service reachable.
• •Roll back to the last known-good revision as a controlled action, after capturing the evidence described above, to restore traffic while the root cause is investigated.
• •Add an explicit "now listening on" log line and a startup self-check that fails the process if the listener does not open within a bounded timeout.

Prove the fix

1. 01An external HTTP probe to the public hostname returns the expected status code consistently across multiple attempts spanning at least one full readiness window after the rollout.
2. 02The orchestrator reports every backing instance as Ready, not just Running, for a full probe interval, and the load balancer or ingress shows all targets as healthy.
3. 03A direct in-network probe to the application port from a second host confirms the listener is open on the address the platform expects, not only on loopback.
4. 04Process logs show a clear listening line within the documented startup window, with no subsequent crashes or restarts for at least one full monitoring cycle.

Prevention and next steps

• •Make the bind address and port externally injected configuration rather than build-time constants, so each environment can override them safely without rebuilding the artifact.
• •Treat the readiness probe as a contract: the application must expose an endpoint whose success exactly matches what the platform probe will check, and the endpoint must be cheap to call.
• •Add a post-deploy synthetic probe that exercises the public path from outside the cluster and alerts on deviation from the previous revision's baseline, so silent reachability loss is detected before users report it.
• •Run a smoke test that opens a TCP connection to the declared port from inside the container as part of startup, failing the container if the listener does not open within a bounded timeout.

Safe commands and checks

curl -sS -o /dev/null -w '%{http_code}\n' --max-time 5 https://<hostname>/<path> — probe the public endpoint and capture the HTTP status code with a bounded timeout.
kubectl get pods -l <label-selector> -o wide — list backing pods with their node, status, and age to confirm they are Running and distributed.
kubectl describe pod <pod-name> | sed -n '/Conditions:/,/Events:/p' — read readiness and liveness conditions plus events for the pod without modifying state.
kubectl get endpoints <service-name> -o yaml — inspect which pod IPs and ports the service is currently routing to.
pgrep -f <process-name> — obtain <pid> for the application process before the next command.
ss -ltnp | grep <pid> — list TCP listeners owned by the application process to verify the bound address and port.
kubectl logs <pod-name> --since=15m — read recent container logs without restarting or rolling the pod.
dig +short <hostname> — resolve the public hostname to its current addresses to confirm DNS points to the expected backend.