---
# === IDENTITY ===
id: software/debugging/kubernetes-crashloopbackoff/2026
canonical_question: "How do I debug Kubernetes CrashLoopBackOff?"
aliases:
  - "Kubernetes CrashLoopBackOff"
  - "K8s pod keeps restarting"
  - "Kubernetes pod crash loop"
  - "kubectl CrashLoopBackOff fix"
  - "Kubernetes container keeps crashing"
  - "K8s pod restart loop"
  - "Kubernetes backoff restarting failed container"
  - "Kubernetes pod exit code 1 137"
entity_type: software_reference
domain: software > debugging > kubernetes_crashloopbackoff
region: global
jurisdiction: global
temporal_scope: 2020-2026

# === VERIFICATION ===
last_verified: 2026-05-17
confidence: 0.94
version: 2.1
first_published: 2026-02-18

# === TEMPORAL VALIDITY ===
temporal_validity:
  status: stable
  last_breaking_change: "K8s 1.33 (Apr 2025) — sidecar containers GA; ReduceDefaultCrashLoopBackOffDecay alpha (1s initial / 60s max backoff)"
  next_review: 2026-11-13
  change_sensitivity: medium

# === CONSTRAINTS ===
constraints:
  - "kubectl logs --previous only retrieves logs from the last crashed container instance — earlier crash logs require centralized logging (ELK, Loki, Datadog)"
  - "Never delete a CrashLoopBackOff pod without first capturing logs and events — deletion destroys diagnostic data"
  - "startupProbe disables liveness and readiness probes until it succeeds — a failing startup probe prevents the pod from ever being live"
  - "Resource limits are enforced at runtime by cgroups — setting memory limits below the app's baseline guarantees OOMKill (exit 137)"
  - "Init containers run sequentially; if any init container fails, ALL init containers restart from the beginning"
  - "Sidecar containers (restartPolicy: Always init containers, K8s 1.29+ GA) follow different lifecycle rules — they survive main container restarts and must be debugged separately"

# === SKIP CONDITIONS ===
skip_this_unit_if:
  - condition: "Pod is stuck in Pending state (never started)"
    use_instead: "software/debugging/kubernetes-pod-pending/2026"
  - condition: "Pod shows ImagePullBackOff (image cannot be pulled)"
    use_instead: "software/debugging/kubernetes-imagepullbackoff/2026"
  - condition: "Container is OOMKilled but the issue is Docker memory limits, not Kubernetes"
    use_instead: "software/debugging/docker-oomkilled/2026"

# === AGENT HINTS ===
inputs_needed:
  - key: exit_code
    question: "What exit code does the crashed container show? (Run: kubectl get pod <pod> -o jsonpath='{.status.containerStatuses[0].lastState.terminated.exitCode}')"
    type: choice
    options: ["0", "1", "126", "127", "137", "143", "Other/Unknown"]
  - key: k8s_version
    question: "What Kubernetes version is the cluster running? (Run: kubectl version --short)"
    type: choice
    options: ["1.28 or older", "1.29-1.31", "1.32+", "Unknown"]
  - key: workload_type
    question: "What kind of workload is crashing?"
    type: choice
    options: ["Deployment/ReplicaSet", "StatefulSet", "DaemonSet", "Job/CronJob", "Standalone Pod"]

# === DISTRIBUTION ===
canonical_source: "https://knowledgelib.io/software/debugging/kubernetes-crashloopbackoff/2026"
suggested_citation: "Source: knowledgelib.io — AI Knowledge Library (verified 2026-05-17)"

# === RELATED UNITS ===
related_kos:
  related_to:
    - id: "software/debugging/docker-oomkilled/2026"
      label: "Docker OOMKilled Debugging"
    - id: "software/debugging/docker-container-wont-start/2026"
      label: "Docker Container Won't Start"
  solves:
    - id: "software/debugging/kubernetes-pod-pending/2026"
      label: "Kubernetes Pod Pending (often follows after scheduling is resolved)"
  often_confused_with:
    - id: "software/debugging/kubernetes-pod-pending/2026"
      label: "Kubernetes Pod Pending (pod never started vs. pod keeps crashing)"
    - id: "software/debugging/nodejs-econnrefused/2026"
      label: "Node.js ECONNREFUSED (app-level vs. orchestration-level error)"

# === SOURCES (11 authoritative sources) ===
sources:
  - id: src1
    title: "Kubernetes — Debugging Pods"
    author: Kubernetes
    url: https://kubernetes.io/docs/tasks/debug/debug-application/debug-pods/
    type: official_docs
    published: 2025-06-01
    reliability: authoritative
  - id: src2
    title: "Kubernetes — Configure Liveness, Readiness and Startup Probes"
    author: Kubernetes
    url: https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/
    type: official_docs
    published: 2025-06-01
    reliability: authoritative
  - id: src3
    title: "Komodor — Kubernetes CrashLoopBackOff Guide"
    author: Komodor
    url: https://komodor.com/learn/how-to-fix-crashloopbackoff-kubernetes-error/
    type: technical_blog
    published: 2025-01-01
    reliability: high
  - id: src4
    title: "Groundcover — CrashLoopBackOff Debugging"
    author: Groundcover
    url: https://www.groundcover.com/blog/kubernetes-crashloopbackoff
    type: technical_blog
    published: 2024-06-01
    reliability: high
  - id: src5
    title: "Atmosly — CrashLoopBackOff Troubleshooting (2025)"
    author: Atmosly
    url: https://atmosly.com/blog/crashloopbackoff-in-kubernetes-causes-solutions-2025
    type: technical_blog
    published: 2025-01-01
    reliability: high
  - id: src6
    title: "KodeKloud — CrashLoopBackOff Fix"
    author: KodeKloud
    url: https://kodekloud.com/blog/crashloopbackoff-fix/
    type: technical_blog
    published: 2024-06-01
    reliability: high
  - id: src7
    title: "Kubernetes — Init Containers"
    author: Kubernetes
    url: https://kubernetes.io/docs/concepts/workloads/pods/init-containers/
    type: official_docs
    published: 2025-06-01
    reliability: authoritative
  - id: src8
    title: "Kubernetes v1.32 Release — KubeletCrashLoopBackOffMax"
    author: Kubernetes
    url: https://kubernetes.io/blog/2024/12/11/kubernetes-v1-32-release/
    type: official_docs
    published: 2024-12-11
    reliability: authoritative
  - id: src9
    title: "Google Kubernetes Engine — Troubleshoot CrashLoopBackOff events"
    author: Google Cloud
    url: https://docs.cloud.google.com/kubernetes-engine/docs/troubleshooting/crashloopbackoff-events
    type: official_docs
    published: 2026-01-15
    reliability: authoritative
  - id: src10
    title: "Kubernetes 1.33 — Sidecar Containers Graduate to GA"
    author: Kubernetes
    url: https://kubernetes.io/blog/2025/04/23/kubernetes-v1-33-release/
    type: official_docs
    published: 2025-04-23
    reliability: authoritative
  - id: src11
    title: "Sysdig — Debug Kubernetes CrashLoopBackOff"
    author: Sysdig
    url: https://www.sysdig.com/blog/debug-kubernetes-crashloopbackoff
    type: technical_blog
    published: 2025-09-01
    reliability: high
---

# How to Debug Kubernetes CrashLoopBackOff

## TL;DR

- **Bottom line**: CrashLoopBackOff means a container in a pod keeps crashing and Kubernetes keeps restarting it with increasing delays (10s, 20s, 40s, ... up to 5 minutes). It's not an error itself — it's Kubernetes telling you the container can't stay running. The root cause is always in the container: app error, bad config, missing dependency, OOM, or failing health probes.
- **Key tool/command**: `kubectl describe pod <pod>` shows events, exit codes, and probe config. `kubectl logs <pod> --previous` shows the crashed container's last output. These two commands solve 90% of CrashLoopBackOff cases.
- **Watch out for**: Liveness probes that are too aggressive — if `initialDelaySeconds` is too short for a slow-starting app, Kubernetes kills the container before it's ready, causing an infinite restart loop. Use `startupProbe` instead for slow starters.
- **Works with**: All Kubernetes versions (1.20+). Same concepts apply to OpenShift, EKS, GKE, AKS, k3s, minikube. K8s 1.32+ adds configurable max backoff via `KubeletCrashLoopBackOffMax`. **K8s 1.33+** (Apr 2025) reduces the default backoff to 1s initial / 60s max under the `ReduceDefaultCrashLoopBackOffDecay` alpha feature gate, and graduates **sidecar containers to GA** [src10].

## Constraints
<!-- Agents: read this section before recommending any action from this unit.
     These are hard constraints that must not be violated. -->

- **Capture before deleting**: Never delete a CrashLoopBackOff pod without first running `kubectl logs --previous` and `kubectl describe pod` — deletion permanently destroys crash logs and events. [src1]
- **Logs are ephemeral**: `kubectl logs --previous` only retrieves the last crash. Earlier crash logs require centralized logging (ELK, Loki, Datadog). [src1, src3]
- **startupProbe blocks other probes**: When a `startupProbe` is configured, it disables liveness and readiness probes until it succeeds. If the startup probe never passes, the pod enters CrashLoopBackOff with no liveness/readiness evaluation. [src2]
- **Memory limits are hard cgroups enforcement**: Setting `resources.limits.memory` below an app's baseline working set guarantees OOMKill (exit 137). Always profile memory usage with `kubectl top pod` before setting limits. [src3, src5]
- **Init containers restart as a group**: If any init container fails, ALL init containers restart from the beginning — not just the failed one. [src7]
- **Sidecar containers have separate lifecycle**: Sidecar containers (K8s 1.29+ GA, defined as init containers with `restartPolicy: Always`) survive main container restarts. Debug them separately with `kubectl logs -c <sidecar-name>`. [src1, src7]

## Quick Reference

| # | Cause | Likelihood | Key Signal | Fix |
|---|---|---|---|---|
| 1 | Application error / crash | ~30% | Exit code 1; error in logs | Fix application code; check dependencies [src1, src3] |
| 2 | Missing/wrong env vars or config | ~20% | Exit code 1; "env var not set" in logs | Fix ConfigMap/Secret/env var configuration [src3, src4] |
| 3 | OOMKilled (exit code 137) | ~15% | `OOMKilled: true` in describe output | Increase `resources.limits.memory` [src3, src5] |
| 4 | Liveness probe failure | ~10% | "Liveness probe failed" in events | Increase `initialDelaySeconds` or use `startupProbe` [src2, src3] |
| 5 | Command not found (exit 127) | ~5% | "exec format error" or "not found" | Fix CMD/entrypoint in Dockerfile; check image [src4, src6] |
| 6 | Missing ConfigMap or Secret | ~5% | "CreateContainerConfigError" then crash | Create the missing ConfigMap/Secret [src3, src5] |
| 7 | Init container failure | ~4% | Init container stuck or crashing | Debug init container: `kubectl logs -c init-name` [src7] |
| 8 | Volume mount failure | ~4% | "MountVolume.SetUp failed" in events | Fix PVC claim, storage class, or permissions [src3, src5] |
| 9 | Permission denied (exit 126) | ~3% | "Permission denied" in logs | Fix file permissions or `securityContext` [src4] |
| 10 | Image pull issues (then crash) | ~2% | "ImagePullBackOff" then crash | Fix image name, tag, or registry credentials [src1, src3] |
| 11 | Container exits successfully (exit 0) | ~2% | Repeated exit 0 restarts | App shouldn't exit; add long-running process or use Job [src5, src6] |

## Decision Tree

> Full script: [decision-tree.txt](scripts/decision-tree.txt) (34 lines)

```
START — Pod shows CrashLoopBackOff
├── Run: kubectl describe pod <pod-name> -n <namespace>
│   ├── Check "Last State" → Exit Code?
│   │   ├── Exit 0 → Container exited successfully but shouldn't
│   │   │   └── App completes and exits → needs long-running process or use Job [src6]
│   │   ├── Exit 1 → Application error
│   │   │   └── kubectl logs --previous → read stack trace / error message [src1, src3]
│   │   ├── Exit 126 → Permission denied
│   │   │   └── Fix file permissions or securityContext.runAsUser [src4]
│   │   ├── Exit 127 → Command/binary not found
│   │   │   └── Fix CMD/ENTRYPOINT; check image architecture (amd64 vs arm64) [src4, src6]
│   │   ├── Exit 137 → Killed (SIGKILL)
│   │   │   ├── Reason: OOMKilled → increase resources.limits.memory [src3, src5]
│   │   │   └── Reason: other → check liveness probe timing or external kill [src2]
│   │   └── Exit 143 → SIGTERM received
│   │       └── Check preStop hook, terminationGracePeriodSeconds, or probe config [src2]
│   ├── Check Events section
│   │   ├── "Liveness probe failed" → fix probe config: use startupProbe [src2, src3]
│   │   ├── "MountVolume.SetUp failed" → fix PVC, storage class, or permissions [src3]
│   │   └── "configmap/secret not found" → create the missing resource [src3, src5]
│   └── Check Init Containers
│       └── Init container failing? → kubectl logs -c <init-name> [src7]
└── kubectl logs --previous → find the specific error message [src1]
```

## Step-by-Step Guide

### 1. Get the pod status and restart count

Start by identifying which pods are in CrashLoopBackOff and how many times they've restarted. [src1, src3]

```bash
# List all pods (all namespaces)
kubectl get pods --all-namespaces | grep CrashLoopBackOff

# Specific namespace
kubectl get pods -n <namespace>

# With wide output (shows node assignment)
kubectl get pods -o wide

# JSON output for scripting
kubectl get pod <pod-name> -o jsonpath='{.status.containerStatuses[0].restartCount}'
```

**Verify**: You can see which pods are affected and their restart counts.

### 2. Describe the pod for events and state

`kubectl describe pod` is the single most important debugging command. [src1, src3, src5]

```bash
# Full pod description
kubectl describe pod <pod-name> -n <namespace>

# Key sections to look at:
# 1. "Last State" — shows exit code and reason
# 2. "Events" — shows what Kubernetes observed
# 3. "Containers" → "State" and "Last State"
# 4. "Conditions" — Ready, ContainersReady, etc.

# Quick exit code check
kubectl get pod <pod-name> -o jsonpath='{.status.containerStatuses[0].lastState.terminated.exitCode}'

# Check if OOMKilled
kubectl get pod <pod-name> -o jsonpath='{.status.containerStatuses[0].lastState.terminated.reason}'
```

**Verify**: The exit code and events tell you the category (app error, OOM, probe failure, config error).

### 3. Read the previous container's logs

The current container may have just started; the *previous* one has the crash output. [src1, src3, src5]

```bash
# Previous (crashed) container's logs
kubectl logs <pod-name> --previous

# Current container's logs
kubectl logs <pod-name>

# Specific container in multi-container pod
kubectl logs <pod-name> -c <container-name> --previous

# Follow logs in real-time
kubectl logs -f <pod-name>

# Init container logs
kubectl logs <pod-name> -c <init-container-name>

# Sidecar container logs (K8s 1.29+)
kubectl logs <pod-name> -c <sidecar-name>

# Last N lines
kubectl logs <pod-name> --previous --tail=50
```

**Verify**: The error message, stack trace, or missing config tells you exactly what to fix.

### 4. Fix OOMKilled (exit code 137)

Container exceeded its memory limit. [src3, src5]

```yaml
# Bad — memory limit too low
resources:
  limits:
    memory: 128Mi  # Too small for a Java app!

# Fixed — increase memory limits based on observed usage
resources:
  requests:
    memory: 256Mi
    cpu: 100m
  limits:
    memory: 512Mi
    cpu: 500m
```

```bash
# Check actual memory usage before adjusting
kubectl top pod <pod-name>

# See resource limits on the pod
kubectl get pod <pod-name> -o jsonpath='{.spec.containers[0].resources}'
```

**Verify**: Pod runs without OOMKilled. Monitor with `kubectl top pod`.

### 5. Fix liveness probe failures

Liveness probes that fire too early or too aggressively kill healthy containers. [src2, src3]

```yaml
# BAD — probe starts immediately; slow app not ready yet
livenessProbe:
  httpGet:
    path: /health
    port: 8080
  initialDelaySeconds: 5    # Too soon for a Spring Boot app!
  periodSeconds: 5
  failureThreshold: 3

# GOOD — use startupProbe for slow starters; relax liveness
startupProbe:
  httpGet:
    path: /health
    port: 8080
  periodSeconds: 10
  failureThreshold: 30        # 30 x 10s = 5 min startup allowance
livenessProbe:
  httpGet:
    path: /health
    port: 8080
  periodSeconds: 15
  failureThreshold: 3
  timeoutSeconds: 5
readinessProbe:
  httpGet:
    path: /ready
    port: 8080
  periodSeconds: 10
  failureThreshold: 3
```

**Verify**: Pod starts without being killed by probes. Events show no "Liveness probe failed".

### 6. Fix missing ConfigMap / Secret / env vars

Missing configuration causes immediate crashes. [src3, src4, src5]

```bash
# Check if ConfigMaps/Secrets exist
kubectl get configmap -n <namespace>
kubectl get secret -n <namespace>

# Describe the specific ConfigMap
kubectl describe configmap <name> -n <namespace>

# Check env vars resolved in the pod spec
kubectl get pod <pod-name> -o jsonpath='{.spec.containers[0].env}' | python -m json.tool

# Check for envFrom references
kubectl get pod <pod-name> -o jsonpath='{.spec.containers[0].envFrom}'
```

```yaml
# Ensure ConfigMap exists and is correctly referenced
apiVersion: v1
kind: ConfigMap
metadata:
  name: app-config
  namespace: default
data:
  DATABASE_URL: "postgres://user:pass@db:5432/mydb"
  LOG_LEVEL: "info"
---
# Pod spec referencing it
env:
  - name: DATABASE_URL
    valueFrom:
      configMapKeyRef:
        name: app-config
        key: DATABASE_URL
# Or load all keys:
envFrom:
  - configMapRef:
      name: app-config
```

**Verify**: ConfigMap/Secret exists and pod can read all referenced keys.

### 7. Debug with ephemeral containers or manual exec

When logs don't show enough, get inside to investigate. [src1, src4]

```bash
# Ephemeral debug container (K8s 1.25+ GA)
kubectl debug -it <pod-name> --image=busybox --target=<container-name>

# If pod has a shell available
kubectl exec -it <pod-name> -- /bin/sh

# Run the image locally to test
docker run -it --entrypoint /bin/sh <image-name>

# Check cluster events for additional context
kubectl get events -n <namespace> --sort-by='.lastTimestamp' | tail -20

# Check node resources (is the node under pressure?)
kubectl top nodes
kubectl describe node <node-name> | grep -A5 "Conditions"
```

**Verify**: You can inspect the container's filesystem, run the command manually, and see the exact error.

## Code Examples

### Comprehensive CrashLoopBackOff diagnostic script

> Full script: [comprehensive-crashloopbackoff-diagnostic-script.sh](scripts/comprehensive-crashloopbackoff-diagnostic-script.sh) (73 lines)

```bash
#!/bin/bash
# Input:  Pod name and namespace
# Output: Complete diagnostic report for CrashLoopBackOff
POD="$1"
NS="${2:-default}"
# ... (see full script)
```

### Well-configured pod with proper probes and resources

> Full script: [well-configured-pod-with-proper-probes-and-resourc.yml](scripts/well-configured-pod-with-proper-probes-and-resourc.yml) (87 lines)

```yaml
# Input:  Pod that keeps crashing due to misconfiguration
# Output: Production-ready pod spec with proper probes, resources, and error handling
apiVersion: apps/v1
kind: Deployment
metadata:
# ... (see full script)
```

### Automated CrashLoopBackOff watcher

> Full script: [automated-crashloopbackoff-watcher.py](scripts/automated-crashloopbackoff-watcher.py) (62 lines)

```python
#!/usr/bin/env python3
# Input:  Kubernetes cluster
# Output: Real-time alerts for CrashLoopBackOff pods with diagnostics
import subprocess
import json
# ... (see full script)
```

## Anti-Patterns

### Wrong: No resource limits

```yaml
# BAD — pod can consume unlimited resources; OOM kills other pods [src3, src5]
containers:
  - name: app
    image: myapp:latest
    # No resources section at all!
```

### Correct: Always set resource requests and limits

```yaml
# GOOD — predictable resource usage; scheduler can place pods properly [src3, src5]
containers:
  - name: app
    image: myapp:1.2.3
    resources:
      requests:
        cpu: 100m
        memory: 256Mi
      limits:
        cpu: 500m
        memory: 512Mi
```

### Wrong: Aggressive liveness probe on slow-starting app

```yaml
# BAD — probe starts too early; kills Spring Boot during startup [src2, src3]
livenessProbe:
  httpGet:
    path: /health
    port: 8080
  initialDelaySeconds: 5     # Spring Boot needs 30-60s to start!
  periodSeconds: 3
  failureThreshold: 3        # Killed after 5 + (3 x 3) = 14 seconds
```

### Correct: Use startupProbe for slow starters

```yaml
# GOOD — startupProbe allows long init; liveness only runs after startup [src2]
startupProbe:
  httpGet:
    path: /health
    port: 8080
  periodSeconds: 10
  failureThreshold: 30       # Up to 5 min startup time
livenessProbe:
  httpGet:
    path: /health
    port: 8080
  periodSeconds: 15
  failureThreshold: 3
```

### Wrong: Using :latest tag

```yaml
# BAD — unpredictable; may pull broken image; hard to rollback [src3, src6]
containers:
  - name: app
    image: myapp:latest       # Which version IS this?
    imagePullPolicy: Always   # Re-pulls every restart
```

### Correct: Pin specific image version

```yaml
# GOOD — reproducible; easy rollback; clear audit trail [src3, src6]
containers:
  - name: app
    image: myapp:1.2.3        # Exact version; matches git tag
    imagePullPolicy: IfNotPresent
```

### Wrong: Ignoring sidecar container lifecycle (K8s 1.29+)

```yaml
# BAD — sidecar defined as regular container; exits when main container restarts [src7]
containers:
  - name: app
    image: myapp:1.2.3
  - name: log-shipper
    image: fluentbit:2.2      # Dies and restarts with main container
```

### Correct: Define sidecars as init containers with restartPolicy: Always

```yaml
# GOOD — sidecar survives main container restarts; proper lifecycle [src7, src8]
initContainers:
  - name: log-shipper
    image: fluentbit:2.2
    restartPolicy: Always     # K8s 1.29+ sidecar pattern
containers:
  - name: app
    image: myapp:1.2.3
```

## Decision Logic

### If `kubectl describe pod` shows `Reason: OOMKilled` (exit 137)
--> The container hit its `resources.limits.memory` ceiling. Profile with `kubectl top pod` first, then raise the memory limit (typically 1.5–2× observed working set). Do NOT just remove the limit — unbounded pods evict their neighbors. [src3, src5, src9]

### If exit code is 1 AND `kubectl logs --previous` shows a stack trace
--> This is an application-level bug, not an orchestration problem. Fix the code path, not the pod spec. Common: unhandled exception during init, missing config value, schema migration not run. [src1, src3, src11]

### If exit code is 0 AND `restartPolicy: Always`
--> The container completed and the controller keeps restarting it. Either (a) the entrypoint is a one-shot task — convert the Deployment to a `Job`/`CronJob`; or (b) the long-running process exited cleanly — keep it in the foreground (no daemonizing, no `&`). [src5, src6, src9]

### If exit code is 137 AND `Reason` is NOT `OOMKilled`
--> The kubelet killed it (usually a failed liveness probe with SIGKILL after grace period). Check `kubectl describe pod` Events for "Liveness probe failed". Fix: add `startupProbe` for slow starters, raise `failureThreshold`, or relax `periodSeconds`. [src2, src3]

### If exit code is 127 ("command not found") OR "exec format error"
--> Either the `CMD`/`ENTRYPOINT` references a binary that isn't in the image, OR there's an image architecture mismatch (amd64 image scheduled on arm64 node). Run `kubectl describe pod` to confirm the node arch, then either rebuild as a multi-arch image or add `nodeSelector: kubernetes.io/arch: amd64`. [src4, src6]

### If you have a sidecar container AND main container keeps crashing
--> **K8s 1.29+ (GA in 1.33)**: sidecar containers (init containers with `restartPolicy: Always`) survive main container restarts and have separate lifecycle. The pod can appear partially ready while the main container is in CrashLoopBackOff. Debug each container individually with `kubectl logs -c <name>`; don't assume pod-level signals reflect the main app. [src1, src7, src10]

### If you're on K8s 1.33+ AND the 5-minute backoff is slowing down recovery
--> Enable the `ReduceDefaultCrashLoopBackOffDecay` alpha feature gate on the kubelet (`--feature-gates=ReduceDefaultCrashLoopBackOffDecay=true`). This caps backoff at 60s instead of 300s, with a 1s initial delay. Per-node override is available via `KubeletCrashLoopBackOffMax` (K8s 1.32+ alpha). Both are alpha — do not enable in production without testing. [src8, src10]

## Common Pitfalls

- **`--previous` flag forgotten**: `kubectl logs <pod>` shows the *current* container's logs, which may be empty if the container just restarted. Always use `--previous` to see what the last crash produced. [src1, src3]
- **Liveness probe killing healthy containers**: If `initialDelaySeconds` is too short, Kubernetes kills the container before it finishes starting. This creates a permanent CrashLoopBackOff where the app never gets to start. Use `startupProbe` (K8s 1.18+) for apps with variable startup times. [src2, src3]
- **Exit 0 CrashLoopBackOff**: A container that exits with code 0 is "successful" but if `restartPolicy: Always` (default for Deployments), Kubernetes restarts it. For one-shot tasks, use a Job instead of a Deployment. [src5, src6]
- **ConfigMap/Secret race condition**: If a pod starts before its ConfigMap or Secret is created, it enters CrashLoopBackOff. Use init containers or ensure configs exist before deployments. [src3, src5]
- **Node resource pressure**: Sometimes CrashLoopBackOff is caused by the node being under memory/disk pressure, evicting pods. Check `kubectl describe node` for "Conditions". [src1]
- **Image architecture mismatch**: Pulling an `amd64` image onto an `arm64` node (or vice versa) causes "exec format error" (exit 1 or 127). Use multi-arch images or node selectors. [src4]
- **Sidecar container masking the real failure**: In K8s 1.29+, sidecar containers survive main container restarts. If the sidecar is healthy but the main container keeps crashing, `kubectl describe pod` may show the pod as partially ready. Check each container status individually. [src7]
- **Backoff timer confusion**: The 5-minute max backoff (300s) means long waits between restart attempts after several failures. Delete and recreate the pod to reset. On K8s 1.32+, administrators can reduce the max backoff via `KubeletCrashLoopBackOffMax` feature gate. [src8]

## Diagnostic Commands

> Full script: [diagnostic-commands.sh](scripts/diagnostic-commands.sh) (28 lines)

```bash
# === Find CrashLoopBackOff pods ===
kubectl get pods -A | grep CrashLoopBackOff
kubectl get pods -A | grep -E "0/[0-9]+"   # Not ready

# === Pod details ===
kubectl describe pod <pod> -n <ns>
kubectl get pod <pod> -o jsonpath='{.status.containerStatuses[0].lastState.terminated.exitCode}'
kubectl get pod <pod> -o jsonpath='{.status.containerStatuses[0].lastState.terminated.reason}'

# === Logs ===
kubectl logs <pod> --previous
kubectl logs <pod> -c <container> --previous
kubectl logs <pod> -c <init-container>

# === Events ===
kubectl get events -n <ns> --sort-by='.lastTimestamp' | tail -20

# === Resources ===
kubectl top pod <pod>
kubectl top nodes

# === Debugging ===
kubectl debug -it <pod> --image=busybox --target=<container>
kubectl exec -it <pod> -- /bin/sh

# === Config ===
kubectl get configmap -n <ns>
kubectl get secret -n <ns>

# === Node health ===
kubectl describe node <node> | grep -A5 "Conditions"
kubectl describe node <node> | grep -A5 "Allocated resources"
```

## Version History & Compatibility

| Version | Behavior | Key Changes |
|---|---|---|
| K8s 1.34 (Aug 2025) | Current | `ContainerRestartRules` alpha (KEP-5307) — per-container `restartPolicyRules` matching exit codes; restart in place even when pod-level `restartPolicy: Never`. Targets AI/ML batch jobs on expensive GPUs [src10] |
| K8s 1.33 (Apr 2025) | Stable | **Sidecar containers GA**; `ReduceDefaultCrashLoopBackOffDecay` alpha — initial backoff 1s, max 60s (down from 10s/300s); kubelet feature gate flag [src10, src11] |
| K8s 1.32 (Dec 2024) | Stable | `KubeletCrashLoopBackOffMax` alpha — configurable max backoff delay (1s-300s per node) [src8] |
| K8s 1.29 (Dec 2023) | Stable | Sidecar containers beta (`restartPolicy: Always` on init containers); better init container lifecycle [src1, src7] |
| K8s 1.28 | Stable | Sidecar containers alpha; improved probe logging [src1] |
| K8s 1.25 | Ephemeral GA | Ephemeral debug containers GA for debugging [src1] |
| K8s 1.23 | Debug beta | `kubectl debug` enters beta [src1] |
| K8s 1.20 | startupProbe GA | `startupProbe` graduated to stable [src2] |
| K8s 1.18 | startupProbe beta | startupProbe for slow-starting containers [src2] |
| K8s 1.16 | Stable probes | Liveness/readiness probes stable in all workloads [src2] |

## When to Use / When Not to Use

| Use When | Don't Use When | Use Instead |
|---|---|---|
| Pod shows CrashLoopBackOff status | Pod is stuck in Pending | Debug scheduling: `kubectl describe pod` → Events |
| Container keeps restarting (high count) | Pod is in ImagePullBackOff | Fix image name/registry/auth |
| Exit code is non-zero | Pod is running but not ready | Debug readiness probe |
| Events show probe failures | Container is Evicted | Check node pressure: `kubectl describe node` |

## Important Caveats

- The backoff delay doubles each restart: 10s → 20s → 40s → 80s → 160s → 300s (max 5 min). This means after several restarts, there's a long delay before the next attempt. You can delete and recreate the pod to reset the backoff timer. On K8s 1.32+, cluster administrators can lower the max via `KubeletCrashLoopBackOffMax` feature gate.
- `kubectl logs --previous` only shows logs from the *last* crash. If you need logs from earlier crashes, use a centralized logging solution (ELK, Loki, Datadog).
- Resource `requests` affect scheduling; `limits` affect runtime enforcement. Set `requests` based on normal usage and `limits` based on peak + headroom. Don't set them equal unless you need QoS class "Guaranteed".
- `startupProbe` disables liveness and readiness probes until it succeeds. This means a failing startup probe prevents the pod from ever being "live", eventually leading to CrashLoopBackOff if `failureThreshold` is exceeded.
- Init containers run sequentially. If any init container fails, the pod restarts all init containers from the beginning. Debug init containers with `kubectl logs -c <init-container-name>`.
- In production, always use `resources.requests` and `resources.limits`. Without them, pods compete for resources unpredictably, and the scheduler can't make good placement decisions.
- K8s 1.34 (Aug 2025) introduced `ContainerRestartRules` alpha (KEP-5307) — per-container `restartPolicyRules` keyed off exit codes that restart in place even when the pod's `restartPolicy: Never`. Targets AI/ML batch jobs on expensive GPUs where full pod rescheduling is prohibitive. Enable via `--feature-gates=ContainerRestartRules=true`. [src10]
- K8s 1.33 (Apr 2025) added the `ReduceDefaultCrashLoopBackOffDecay` alpha feature gate. When enabled on the kubelet, restart backoff starts at 1s (was 10s) and caps at 60s (was 300s). Sequence becomes 1s → 2s → 4s → 8s → 16s → 32s → 60s. Recovery is ~5× faster but failing pods burn more CPU. [src10, src11]

## Related Units
<!-- Generated from related_kos frontmatter -->

- [Docker OOMKilled Debugging](/software/debugging/docker-oomkilled/2026)
- [Docker Container Won't Start](/software/debugging/docker-container-wont-start/2026)
- [Kubernetes Pod Pending](/software/debugging/kubernetes-pod-pending/2026)
- [Node.js ECONNREFUSED](/software/debugging/nodejs-econnrefused/2026)