Container network debugging checklist

The symptoms

• •Application inside the container logs a connection refused, timeout, no route to host, or DNS resolution error when contacting a peer service or external endpoint.
• •curl, wget, or SDK client inside the container exits non-zero against the expected peer hostname or IP, while the same client works from the Docker host or another container on a different network.
• •Outbound TCP or UDP from the container never reaches the peer, observed as zero bytes on tcpdump or stalled connect attempts in strace.
• •Inbound traffic reaches the wrong container, observed as the host port mapping to a different container ID than expected, or a published port answering with the wrong service banner.
• •Intermittent failures after container restart or after attaching/detaching networks, suggesting stale name resolution or duplicated addresses.

Likely causes

• •Container is attached to the wrong user-defined bridge network, or attached to none, so it can reaches peers only via the default bridge and not via the intended overlay or custom bridge.
• •DNS inside the container resolves the peer service name to an address on a network the container is not attached to, or to an address outside the configured subnet, indicating missing or wrong network attachment.
• •Peer service listens on a different port than the client expects, or binds to a loopback inside its own container so it is unreachable across the container boundary.
• •Published host port maps to a different container than intended because multiple containers publish overlapping ports, producing address collision on the host.
• •Firewall, iptables, or routing on the host blocks the container's bridge or overlay subnet, so packets leave the container but never arrive at the peer or return.

First ten minutes

1. 01Confirm scope: identify the failing container by name or ID and the exact peer it cannot reach (service name, IP, port) before touching the network; record both as evidence.
2. 02Inspect the container's network attachments with docker inspect on the NetworkSettings.Networks field and list the networks it is actually attached to versus the ones peers advertise.
3. 03Reproduce the failure from inside the container using the same user, command, and environment as the application; capture the exact error string so later fixes can be correlated.
4. 04Test DNS independently of the application by resolving the peer service name from inside the container and comparing the returned address to the peer's container IP and subnet.
5. 05Test reachability at L4 independently of DNS by issuing a connect to the resolved IP and port and observing whether the TCP handshake completes, times out, or is refused.

Evidence to collect

• •docker inspect output showing NetworkSettings.Networks, IPAddress, and Gateway for the failing container and the peer container.
• •docker network inspect output showing the subnet, gateway, and container membership list for the relevant user-defined network.
• •DNS resolution result from inside the container, including the question, the answer section, and the server used; flag results that differ from the peer's documented IP.
• •TCP connect result against the peer's IP and port, distinguishing refused (RST), timeout (no SYN-ACK), and success; for published-port mismatches, also collect docker port output.
• •Application log lines from inside the container for the failing request, including timestamps aligned to the diagnostic commands.

Where to look

• •The container-to-container boundary defined by the user-defined bridge or overlay network the failing container is attached to, including its subnet and gateway.
• •The container's embedded DNS resolver path, which is normally the user-defined network's nameserver, versus the upstream resolvers configured on the Docker host.
• •The peer service's listening interface inside its own container, since a loopback-only bind blocks cross-container reachability regardless of network attachment.
• •The host port publishing boundary on the Docker host, where docker port and iptables NAT rules map host:port to container:port and can collide between containers.
• •Host firewall and routing tables that may filter or blackhole the bridge or overlay subnet even when Docker networking itself is correctly configured.

Diagnostic steps

1. 01List the failing container's networks with docker inspect --format and confirm at least one matches a network the peer container is also attached to; a mismatch here is a sufficient cause and ends further network-layer checks.
2. 02Resolve the peer service name from inside the failing container using getent hosts or nslookup and compare the returned A record to the peer container's IPAddress from docker inspect; a mismatch indicates DNS or aliasing misconfiguration rather than connectivity loss.
3. 03Connect to the peer's IP and port using a generic TCP probe (for example, a short-lived netcat or a language-specific socket open with a timeout); classify the result as refused, timeout, or success to separate routing, filtering, and application-layer causes.
4. 04If the peer is reached via a published host port, run docker port on the peer container to confirm host:port maps to container:port and that no other container publishes the same host:port; collisions here point to the wrong-container-reached symptom.
5. 05Capture the container's view of routes and DNS with docker exec combined with ip route and cat of /etc/resolv.conf, comparing the nameserver to the user-defined network's gateway; container-supplied resolvers differ from host resolvers by design.
6. 06If L4 reachability succeeds but the application still fails, inspect the peer process inside its container to confirm it binds to a non-loopback interface on the expected port; loopback binds are a common silent cause of cross-container failure.

Common mistakes

• •Assuming a container can reach a peer because both are "on Docker"; reachability requires shared attachment to the same user-defined network, which the default bridge does not provide for service-name DNS.
• •Trusting the embedded DNS without verifying the question and answer, because a stale cache or an external resolver override can return an address outside the container's reachable subnets.
• •Reaching the peer by host IP from inside the container, which only works on the default bridge with published ports and fails silently on user-defined bridges where host IPs are not routable.
• •Publishing the same host port from two containers and treating the symptom as application failure, when docker port will show the mapping belongs to a different container ID.
• •Restarting the container as a fix without checking whether the peer binds to loopback inside its container, which restarts cannot resolve and which requires a configuration change on the peer.

Safe fixes

• •If networks do not match, attach the failing container to the same user-defined network as the peer using docker network connect, then re-run the L4 probe; do not detach networks as a substitute, since removing attachments can break unrelated services.
• •If DNS resolves to the wrong address, fix the source of the name (compose service alias, extra_hosts entry, or external resolver) rather than hard-coding IPs in the client; after the fix, re-resolve from inside the container and confirm the new answer matches the peer's IPAddress.
• •If the peer binds only to loopback inside its container, change the peer service's bind address to 0.0.0.0 (or the documented interface) in its configuration and restart only the peer container; verify with a fresh L4 probe from the failing container.
• •If two containers publish overlapping host ports, stop the unintended container or change its port mapping to a free host port, then confirm docker port on the intended container shows the expected host:port mapping before re-testing inbound reachability.
• •If host firewall or routing blocks the subnet, adjust the host rules to permit the user-defined network's subnet rather than disabling firewall globally; re-run the L4 probe from inside the container to confirm packets now arrive.

Prove the fix

1. 01From inside the failing container, resolve the peer service name and confirm the returned A record equals the peer container's IPAddress recorded before the fix.
2. 02From inside the failing container, open a TCP connection to the peer's IP and port and observe a completed handshake within the timeout; record the round-trip time as a baseline.
3. 03From the application, repeat the exact request that previously failed and observe a successful response code and payload; correlate the log line timestamp with the diagnostic probe to confirm causation, not coincidence.
4. 04For inbound cases, run docker port on the intended peer container and confirm the host:port mapping points to that container's ID and not another container's ID; verify from outside the host that the expected service banner returns.
5. 05Stop and start the failing container on the same network and confirm both DNS and L4 probes still succeed, ruling out restart-dependent flapping as the real cause.

Prevention and next steps

• •Define networks in compose or orchestration manifests with explicit names and document which services share each network, so attachment mismatches are visible in code review rather than discovered at runtime.
• •Standardize peer service bind addresses on a non-loopback interface and record the expected listen port alongside the service name, so loopback-bind regressions fail fast in staging.
• •Avoid publishing the same host port from multiple containers; if multiple instances are required, use distinct host ports and a reverse proxy or load balancer in front of them.
• •Add a lightweight readiness check that resolves the peer name and opens a TCP connection to the peer port during container start, so attachment and DNS regressions are caught before traffic is served.
• •Keep host firewall rules under configuration management and version them with the network definitions, so subnet changes are matched by rule changes instead of being silently blocked.

Safe commands and checks

docker inspect --format '{{json .NetworkSettings.Networks}}' <container>
docker network inspect --format '{{json .Containers}} {{json .IPAM.Config}}' <network>
docker exec <container> getent hosts <peer-service-name>
docker exec <container> cat /etc/resolv.conf
docker exec <container> ip route
docker exec <container> sh -c 'exec 3<>/dev/tcp/<peer-ip>/<peer-port> && echo open'
docker port <container>