Kubernetes CrashLoopBackOff checklist

The symptoms

• •Pod STATUS stuck in CrashLoopBackOff with RESTARTS counter incrementing while the container repeatedly exits non-zero within seconds of start.
• •kubectl describe pod shows Last State: Terminated with a non-zero Exit Code and a short Finished timestamp indicating fast failure rather than a long-running crash.
• •Events column displays Back-off restarting failed container with a steadily increasing exponential delay (e.g., 5s, 10s, 20s, 40s) between restart attempts.
• •kubectl get pod output shows READY 0/1 with the container reporting Started: false or Ready: false even after waiting past the restart delay window.
• •Application logs retrieved via kubectl logs --previous end abruptly with stack traces, panic messages, or immediate process exit rather than steady request-serving activity.

Likely causes

• •Missing or invalid configuration values, such as required environment variables, ConfigMap keys, or Secret references, causing the process to exit before binding to its listening port.
• •Image or entrypoint mismatch, where the image tag is wrong, the ENTRYPOINT/CMD is malformed, or the working directory contains a missing script or binary path.
• •Readiness or liveness probe misconfiguration where the probe path, port, scheme, or initialDelaySeconds does not match the application's actual startup behavior, so a running container is repeatedly killed.
• •Resource starvation at startup, where the container's requests for CPU or memory are below what the application needs to initialize, causing an OOMKill on the very first attempts.
• •Downstream dependency failure, such as an unreachable database, DNS resolution failure to an external service, or a service endpoint that has no ready endpoints during early startup.

First ten minutes

1. 01Confirm scope by running kubectl get pods -n <namespace> -l <selector> and recording which pod(s) show CrashLoopBackOff versus ImagePullBackOff or ErrImagePull; CrashLoopBackOff specifically means the image started but exited.
2. 02Run kubectl describe pod <pod> -n <namespace> and read the Events column from oldest to newest to capture the Back-off restarting failed container messages with their timestamps and the Last State exit reason.
3. 03Fetch the current container output with kubectl logs <pod> -n <namespace> --previous --tail=200 to see the last failed attempt, then compare against kubectl logs <pod> -n <namespace> --tail=200 to confirm whether the container has ever logged steady-state activity.
4. 04Decide whether the exit reason in describe (Error, OOMKilled, Completed) points to a startup crash, a resource kill, or a process that simply ran and exited; this determines whether to investigate manifest, resources, or workload design.
5. 05Verify the workload definition with kubectl get pod <pod> -n <namespace> -o yaml and inspect containers[].image, envFrom, volumeMounts, and probes for typos, missing keys, or absolute paths that do not exist inside the image filesystem.
6. 06Temporarily disable suspect probes in a copy of the manifest (do not edit the live workload) to test whether the container reaches Ready without probe interference; revert immediately after observation.

Evidence to collect

• •Exact exit code and exit reason from kubectl describe pod, recorded under Last State > Terminated > Reason and Exit Code, plus the Finished timestamp delta to determine fast-fail versus slow-fail pattern.
• •Full stdout/stderr from the failed attempt via kubectl logs --previous, including any panic lines, stack traces, or messages printed immediately before process exit.
• •Probe configuration from the pod spec: httpGet path and port, tcpSocket port, exec command, initialDelaySeconds, periodSeconds, failureThreshold, and timeoutSeconds, cross-referenced with the application's actual readiness behavior.
• •Resource settings: containers[].resources.requests and limits for CPU and memory, plus the node's allocatable capacity and current usage via kubectl describe node, to evaluate whether OOMKill is plausible.
• •Dependency reachability evidence: Service and Endpoints state for any referenced hostname, DNS resolution inside the pod via a short-lived debug container, and Secret/ConfigMap presence via kubectl get with --ignore-not-found.
• •Event timeline of Back-off restarting failed container messages with their respective restart counts to confirm the loop is exponential and not triggered by node pressure or evictions.

Where to look

• •At the pod boundary: kubectl describe pod <pod> -n <namespace> output, specifically the Conditions, Containers, and Events sections in that order.
• •At the container boundary: kubectl logs <pod> -n <namespace> --previous for the most recent terminated instance output and the live logs for the current attempt.
• •At the workload boundary: the pod spec rendered by kubectl get pod <pod> -n <namespace> -o yaml, particularly the containers, initContainers, and volumes blocks including projected ConfigMap and Secret sources.
• •At the cluster scheduling boundary: kubectl describe node events and the kubelet-reported Conditions, plus any FailedScheduling or NodeAffinity-related messages that could prevent the pod from ever reaching a node.
• •At the dependency boundary: kubectl get endpoints -n <namespace> for backing Services to confirm there are real endpoint addresses, and resolve any external hostnames from inside a debug pod to validate DNS and network reachability.

Diagnostic steps

1. 01Differentiate fast-crash from probe-kill by reading Last State Exit Code: an Error exit within the first few seconds of container start indicates an application startup failure, whereas a later termination with reason Completed or a probe failure message points to probe misconfiguration.
2. 02Compare the container's startup logs against the probe's initialDelaySeconds; if logs show the process binding to its port and serving requests only after the probe has already failed repeatedly, the probe is too aggressive.
3. 03Reproduce locally by running the same image and command outside the cluster with the same environment variables to confirm the failure is environmental rather than cluster-specific.
4. 04Inspect initContainers status in describe; a failed initContainer will surface as a non-zero exit reason before the main container ever starts, so check each initContainer's Last State before assuming the application container is at fault.
5. 05Validate mounted configuration by exec'ing into the pod with kubectl debug or by mounting a debug sidecar to read /etc/config or the configured mount path and confirm expected keys are present and correctly formatted.
6. 06Correlate restart timestamps with node-level pressure events; if restarts coincide with MemoryPressure or DiskPressure transitions on the node, root cause is node capacity rather than application code.
7. 07Use kubectl get events --sort-by=.lastTimestamp in the pod's namespace to map the Back-off restarting failed container events against any ImagePull, FailedMount, or FailedCreatePodSandBox events that may be co-occurring.

Common mistakes

• •Restarting the pod as a first action without reading the previous-instance logs, which destroys the only record of the actual failure reason and obscures whether the exit was a crash or a probe kill.
• •Increasing livenessProbe failureThreshold or initialDelaySeconds blindly, which can mask a real startup bug and only delay the inevitable restart loop while breaking restart-backoff math.
• •Editing the live Deployment while probes are still misconfigured, which causes rolling updates that compound the loop across multiple replicas and makes the events timeline harder to read.
• •Assuming CrashLoopBackOff always means an application bug; the loop can also be triggered by missing ConfigMaps/Secrets, initContainer failures, or node-level resource exhaustion.
• •Confusing Ready: false due to a failing readiness probe with CrashLoopBackOff; readiness failures typically do not increment RESTARTS and the container stays Running, not in the CrashLoopBackOff status.
• •Logging into the container with a shell to inspect state when the container exits so quickly that any session terminates immediately; use kubectl debug or read logs and manifests instead.

Safe fixes

• •If the previous logs show a missing file or environment variable, patch the workload via a copy of the manifest to add the required Secret/ConfigMap and env entries, then apply it to a non-production replica set first; verify the new pod reaches Ready: true and RESTARTS stays at zero for at least one probe period.
• •If the exit reason is OOMKilled, raise the memory limit in the workload copy only after confirming via metrics that the application legitimately needs more memory at startup; verify by observing the new pod remains Running across at least three probe cycles with stable memory usage below the new limit.
• •If a readiness or liveness probe is mismatched, adjust the probe definition in a manifest copy to point at the correct path/port and set initialDelaySeconds to at least the observed application startup time; verify by checking that the container reaches Ready without RESTARTS incrementing over a window equal to several periodSeconds.
• •If an initContainer is failing, fix the initContainer command or image in a manifest copy before the main container can ever succeed; verify by confirming all initContainers report Ready and the main container's Last State transitions to Running.
• •If downstream endpoints are missing, fix the backing Service selector or scale the dependent workload; verify via kubectl get endpoints that endpoint addresses exist and a debug resolution returns the expected cluster IP.

Prove the fix

1. 01Confirm kubectl get pod <pod> -n <namespace> shows STATUS Running, READY 1/1, and RESTARTS unchanged from the moment the fix was applied for a window of at least 5x the probe periodSeconds.
2. 02Confirm kubectl describe pod <pod> -n <namespace> shows no new Back-off restarting failed container events since the fix timestamp and the Last State section is empty or older than the fix.
3. 03Confirm kubectl logs <pod> -n <namespace> --tail=200 shows steady-state application activity (request handling, health endpoints responding) rather than repeated startup banners or panic traces.
4. 04Confirm the probe behavior matches expectations: kubectl describe pod shows the liveness/readiness probes in the Containers section with results that have transitioned to Success at least once after the fix.
5. 05Confirm that scaling the same fixed manifest to additional replicas also yields Running and Ready pods with zero restarts, proving the fix is not replica-specific.

Prevention and next steps

• •Adopt a startup probe (startupProbe) for slow-starting applications so liveness checks do not compete with initialization, and align initialDelaySeconds with measured cold-start time.
• •Pin image digests rather than mutable tags in production manifests so a rebuilt image with a broken entrypoint cannot be pulled silently and trigger an unexpected crash loop.
• •Enforce pre-deployment validation of required ConfigMap and Secret keys via admission policies or CI schema checks so missing configuration surfaces before the pod ever starts.
• •Set resource requests and limits based on measured startup and steady-state usage, and monitor OOMKilled events across namespaces to detect starvation before it manifests as a loop.
• •Keep initContainers minimal and idempotent, and treat any change to init container commands as a release-blocking change requiring the same review as main container changes.

Safe commands and checks

kubectl get pods -n <namespace> -l <selector> -o wide
kubectl describe pod <pod> -n <namespace>
kubectl logs <pod> -n <namespace> --previous --tail=200
kubectl logs <pod> -n <namespace> --tail=200
kubectl get pod <pod> -n <namespace> -o yaml
kubectl get events -n <namespace> --sort-by=.lastTimestamp
kubectl get endpoints -n <namespace> <service>
kubectl describe node <node> | grep -A20 Conditions