---
# === IDENTITY ===
id: software/debugging/kubernetes-pod-pending/2026
canonical_question: "How do I debug Kubernetes pods stuck in Pending state?"
aliases:
  - "Kubernetes pod Pending"
  - "K8s pod stuck Pending"
  - "Kubernetes pod not scheduling"
  - "kubectl pod Pending fix"
  - "Kubernetes insufficient resources"
  - "Kubernetes node selector no match"
  - "Kubernetes taint toleration pod"
  - "Kubernetes PVC Pending"
  - "Kubernetes FailedScheduling"
  - "Kubernetes pod waiting to schedule"
entity_type: software_reference
domain: software > debugging > kubernetes_pod_pending
region: global
jurisdiction: global
temporal_scope: 2020-2026

# === VERIFICATION ===
last_verified: 2026-05-17
confidence: 0.94
version: 2.1
first_published: 2026-02-18

# === TEMPORAL VALIDITY ===
temporal_validity:
  status: stable
  last_breaking_change: "K8s 1.36 — Gang Scheduling Beta + Workload-Aware Preemption Alpha (2026-04-22)"
  next_review: 2026-11-13
  change_sensitivity: low

# === CONSTRAINTS ===
constraints:
  - "Scheduling uses resource *requests* not limits — a pod requesting 4Gi is unschedulable on a node with 3Gi allocatable even if actual usage is low"
  - "Never delete a Pending pod to 'fix' scheduling — delete the root cause (taint, quota, missing label) or the pod will just Pending again"
  - "WaitForFirstConsumer PVCs are zone-aware — PV in zone-a + only available node in zone-b = both stay Pending"
  - "On managed K8s (EKS/GKE/AKS), wait 1-5 min for Cluster Autoscaler before reducing requests — a Pending pod triggers node scale-up"
  - "In-Place Pod Resize (GA in K8s 1.35) can adjust CPU/memory on running pods but cannot fix scheduling — already-Pending pods must still wait for node capacity"
  - "RequiredDuringScheduling anti-affinity with more replicas than nodes is permanently unsatisfiable — use PreferredDuringScheduling instead"
  - "On K8s 1.36+, a PodGroup using gang scheduling stays Pending until ALL members fit — partial-binding is rejected by design; check 'kubectl describe podgroup' before debugging individual pods"

# === SKIP CONDITIONS ===
skip_this_unit_if:
  - condition: "Pod shows CrashLoopBackOff instead of Pending"
    use_instead: "software/debugging/kubernetes-crashloopbackoff/2026"
  - condition: "Pod shows ContainerCreating (image pull or volume mount issue)"
    use_instead: "software/debugging/kubernetes-container-creating/2026"
  - condition: "Pod is Running but not Ready (readiness probe failure)"
    use_instead: "software/debugging/kubernetes-pod-not-ready/2026"

# === AGENT HINTS ===
inputs_needed:
  - key: scheduler_event
    question: "What does 'kubectl describe pod <pod>' show in the Events section?"
    type: text
  - key: cluster_type
    question: "What type of Kubernetes cluster is this?"
    type: choice
    options: ["EKS", "GKE", "AKS", "Self-managed", "k3s/minikube/kind", "OpenShift"]
  - key: pending_duration
    question: "How long has the pod been Pending?"
    type: choice
    options: ["Under 5 minutes", "5-30 minutes", "Over 30 minutes"]

# === DISTRIBUTION ===
canonical_source: "https://knowledgelib.io/software/debugging/kubernetes-pod-pending/2026"
suggested_citation: "Source: knowledgelib.io — AI Knowledge Library (verified 2026-05-17)"

# === RELATED UNITS ===
related_kos:
  related_to:
    - id: "software/debugging/kubernetes-crashloopbackoff/2026"
      label: "Kubernetes CrashLoopBackOff Debugging"
    - id: "software/debugging/docker-container-wont-start/2026"
      label: "Docker Container Won't Start"
  solves:
    - id: "software/debugging/docker-oomkilled/2026"
      label: "Docker OOMKilled (related resource constraint)"
  often_confused_with:
    - id: "software/debugging/kubernetes-crashloopbackoff/2026"
      label: "Kubernetes CrashLoopBackOff (container crashed after scheduling, not stuck before)"

# === SOURCES (5-8 authoritative sources) ===
sources:
  - id: src1
    title: "Kubernetes — Debugging Pods"
    author: Kubernetes
    url: https://kubernetes.io/docs/tasks/debug/debug-application/debug-pods/
    type: official_docs
    published: 2025-12-17
    reliability: authoritative
  - id: src2
    title: "Kubernetes — Taints and Tolerations"
    author: Kubernetes
    url: https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/
    type: official_docs
    published: 2025-12-17
    reliability: authoritative
  - id: src3
    title: "Komodor — Pod Pending Troubleshooting"
    author: Komodor
    url: https://komodor.com/learn/pod-in-pending-state-top-6-causes-and-how-to-resolve/
    type: technical_blog
    published: 2025-06-01
    reliability: high
  - id: src4
    title: "Datadog — Kubernetes Pending Pods: How to Debug"
    author: Datadog
    url: https://www.datadoghq.com/blog/debug-kubernetes-pending-pods/
    type: technical_blog
    published: 2025-09-01
    reliability: high
  - id: src5
    title: "Groundcover — Fixing Kubernetes Pod Pending: Causes & Solutions"
    author: Groundcover
    url: https://www.groundcover.com/blog/fixing-kubernetes-pod-pending-causes-troubleshooting-steps
    type: technical_blog
    published: 2025-06-01
    reliability: high
  - id: src6
    title: "Kubernetes — Persistent Volumes"
    author: Kubernetes
    url: https://kubernetes.io/docs/concepts/storage/persistent-volumes/
    type: official_docs
    published: 2025-12-17
    reliability: authoritative
  - id: src7
    title: "Kubernetes — Assigning Pods to Nodes"
    author: Kubernetes
    url: https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/
    type: official_docs
    published: 2025-12-17
    reliability: authoritative
  - id: src8
    title: "Kubernetes v1.35 — In-Place Pod Resize GA"
    author: Kubernetes
    url: https://kubernetes.io/blog/2025/12/19/kubernetes-v1-35-in-place-pod-resize-ga/
    type: official_docs
    published: 2025-12-19
    reliability: authoritative
  - id: src9
    title: "Kubernetes v1.36 — Gang Scheduling Beta + Workload-Aware Preemption"
    author: Kubernetes
    url: https://kubernetes.io/docs/concepts/scheduling-eviction/gang-scheduling/
    type: official_docs
    published: 2026-04-22
    reliability: authoritative
---

# How to Debug Kubernetes Pods Stuck in Pending State

## TL;DR

- **Bottom line**: A Pending pod means the Kubernetes scheduler accepted it but can't find a node to place it on. The most common causes are: insufficient CPU/memory on existing nodes (~35%), taints without matching tolerations (~20%), PVC binding failures (~15%), and nodeSelector/affinity mismatches (~12%). The Events section in `kubectl describe pod` always tells you why.
- **Key tool/command**: `kubectl describe pod <pod> -n <ns>` — the Events section at the bottom shows the exact scheduler reason (e.g., "0/3 nodes are available: 1 Insufficient cpu, 2 node(s) had taint that the pod didn't tolerate").
- **Watch out for**: Resource *requests* (not limits) drive scheduling. A pod requesting 4Gi memory won't schedule on a node with 3Gi allocatable, even if actual usage is low. Also, `WaitForFirstConsumer` PVCs won't bind until a pod actually needs them — this is normal, not an error.
- **Works with**: All Kubernetes versions (1.20+). Same concepts apply to OpenShift, EKS, GKE, AKS, k3s, minikube. K8s 1.35 added In-Place Pod Resize GA; **K8s 1.36 (April 2026) promotes Gang Scheduling to Beta and adds Workload-Aware Preemption (alpha)** — for AI/HPC workloads, PodGroups now schedule and preempt as a unit so 400-of-512 partial bindings stop wasting GPUs. [src9]

## Constraints
<!-- Agents: read this section before recommending any action from this unit.
     These are hard constraints that must not be violated. -->

- Scheduling uses resource *requests* not limits — a pod requesting 4Gi is unschedulable on a node with 3Gi allocatable even if actual usage is low. [src1, src4]
- Never delete a Pending pod to "fix" scheduling — delete the root cause (taint, quota, missing label) or the pod will just Pending again. [src3]
- `WaitForFirstConsumer` PVCs are zone-aware — PV in zone-a + only available node in zone-b = both stay Pending. [src6]
- On managed K8s (EKS/GKE/AKS), wait 1-5 min for Cluster Autoscaler before reducing requests — a Pending pod triggers node scale-up. [src4]
- In-Place Pod Resize (GA in K8s 1.35) can adjust CPU/memory on running pods but cannot fix scheduling — already-Pending pods must still wait for node capacity. [src8]
- RequiredDuringScheduling anti-affinity with more replicas than nodes is permanently unsatisfiable — use PreferredDuringScheduling instead. [src7]
- On K8s 1.36+, a PodGroup using gang scheduling stays Pending until ALL members fit — partial-binding is rejected by design. Check `kubectl describe podgroup <name>` before debugging individual pods. [src9]

## Quick Reference

| # | Cause | Likelihood | Scheduler Message | Fix |
|---|---|---|---|---|
| 1 | Insufficient CPU/memory | ~35% | "Insufficient cpu" or "Insufficient memory" | Reduce requests, add nodes, or free resources [src1, src4] |
| 2 | Taints without tolerations | ~20% | "node(s) had taint that the pod didn't tolerate" | Add toleration to pod or remove taint from node [src2, src4] |
| 3 | PVC not bound | ~15% | "pod has unbound immediate PersistentVolumeClaims" | Fix PVC: check StorageClass, provision PV [src3, src6] |
| 4 | nodeSelector mismatch | ~12% | "node(s) didn't match Pod's node affinity/selector" | Fix selector or label nodes [src4, src7] |
| 5 | Node affinity mismatch | ~5% | "node(s) didn't match Pod's node affinity/selector" | Fix affinity rules or add matching nodes [src4, src7] |
| 6 | Pod anti-affinity | ~4% | "node(s) didn't match pod anti-affinity rules" | Reduce replicas or add more nodes [src4, src7] |
| 7 | Node cordoned / unschedulable | ~3% | "node(s) were unschedulable" | Uncordon node: `kubectl uncordon <node>` [src3, src4] |
| 8 | ResourceQuota exceeded | ~2% | "exceeded quota" in namespace events | Increase quota or reduce resource requests [src1, src3] |
| 9 | PodDisruptionBudget blocking | ~2% | "disruption budget would be violated" | Adjust PDB minAvailable/maxUnavailable [src3] |
| 10 | Topology spread constraints | ~1% | "node(s) didn't satisfy topology spread" | Relax constraints or add nodes in topology [src7] |
| 11 | Node condition (DiskPressure/MemoryPressure) | ~1% | "node(s) had condition: [condition]" | Resolve node pressure or add healthy nodes [src4, src5] |
| 12 | Scheduler not running | <1% | No events at all | Check kube-scheduler pod in kube-system [src1] |
| 13 | PodGroup gang scheduling waiting (K8s 1.36+) | varies (AI/HPC) | "WaitingForGangScheduling" or partial PodGroup binding | Check `kubectl describe podgroup`; PodGroup waits for ALL members [src9] |

## Decision Tree

> Full script: [decision-tree.txt](scripts/decision-tree.txt) (29 lines)

```
START — Pod stuck in Pending
├── Run: kubectl describe pod <pod> -n <ns>
│   └── Check Events section at the bottom
│       ├── "Insufficient cpu" or "Insufficient memory"
│       │   ├── Check requests vs allocatable: kubectl describe nodes [src1, src4]
│       │   ├── Requests too high? → Reduce requests based on kubectl top data
│       │   ├── No headroom? → Add nodes or evict low-priority pods
│       │   └── K8s 1.35+? → Consider In-Place Pod Resize for running pods [src8]
│       ├── "node(s) had taint that the pod didn't tolerate"
│       │   ├── List taints: kubectl describe nodes | grep Taints [src2]
│       │   ├── Add toleration to pod spec, OR
│       │   └── Remove taint: kubectl taint node <node> <key>-
│       ├── "pod has unbound immediate PersistentVolumeClaims"
│       │   ├── Check PVC: kubectl get pvc -n <ns> [src6]
│       │   ├── StorageClass exists? kubectl get storageclass
│       │   └── WaitForFirstConsumer? → This is normal; pod + PVC co-schedule
│       ├── "didn't match Pod's node affinity/selector"
│       │   ├── Check labels: kubectl get nodes --show-labels [src7]
│       │   ├── Fix: kubectl label node <node> <key>=<value>
│       │   └── Or relax pod affinity/nodeSelector rules
│       ├── "node(s) were unschedulable"
│       │   └── kubectl uncordon <node> [src3]
│       ├── "exceeded quota"
│       │   └── kubectl describe resourcequota -n <ns> [src1]
│       ├── "didn't satisfy topology spread"
│       │   └── Relax maxSkew or use ScheduleAnyway [src7]
│       ├── "node(s) had condition"
│       │   └── Check DiskPressure/MemoryPressure: kubectl describe node [src4]
│       └── No events at all
│           └── Check kube-scheduler is running: kubectl get pods -n kube-system [src1]
└── Still stuck? → kubectl get events -A --sort-by='.lastTimestamp'
```

## Step-by-Step Guide

### 1. Identify the Pending pods

Find which pods are stuck and how long they've been waiting. [src1, src3]

```bash
# List all Pending pods across namespaces
kubectl get pods -A --field-selector=status.phase=Pending

# Specific namespace
kubectl get pods -n <namespace> | grep Pending

# With age and node info
kubectl get pods -o wide | grep Pending

# How long has it been Pending?
kubectl get pod <pod> -o jsonpath='{.metadata.creationTimestamp}'
```

**Verify**: You can see which pods are stuck and in which namespace.

### 2. Read the scheduler events

This is the most important step — scheduler events tell you exactly why. [src1, src3, src4]

```bash
# Full pod description (focus on Events section)
kubectl describe pod <pod> -n <namespace>

# Just the events for this pod
kubectl get events -n <namespace> --field-selector involvedObject.name=<pod> --sort-by='.lastTimestamp'

# Events across the namespace
kubectl get events -n <namespace> --sort-by='.lastTimestamp' | tail -20
```

**Verify**: The Events section contains a message like "0/5 nodes are available: ..." with the specific reason.

### 3. Check node resources (for Insufficient CPU/memory)

Compare what the pod requests against what nodes have available. [src1, src4, src5]

```bash
# Allocatable resources per node
kubectl describe nodes | grep -A5 "Allocatable:"

# Current resource usage per node
kubectl top nodes

# Detailed allocation: see how much is requested vs allocatable
kubectl describe node <node-name> | grep -A20 "Allocated resources"

# Quick summary: all nodes' allocatable CPU and memory
kubectl get nodes -o custom-columns=\
  NAME:.metadata.name,\
  CPU:.status.allocatable.cpu,\
  MEM:.status.allocatable.memory,\
  READY:.status.conditions[-1:].status

# Check what the pod is requesting
kubectl get pod <pod> -o jsonpath='{.spec.containers[*].resources.requests}'
```

**Verify**: Compare pod requests to available resources. If no node has enough, that's the problem.

### 4. Check taints and tolerations

Taints on nodes repel pods that don't have matching tolerations. [src2, src4]

```bash
# List all taints on all nodes
kubectl get nodes -o custom-columns=\
  NAME:.metadata.name,\
  TAINTS:.spec.taints

# Or for each node
kubectl describe node <node> | grep -A5 "Taints:"

# Check the pod's tolerations
kubectl get pod <pod> -o jsonpath='{.spec.tolerations}' | python -m json.tool

# Remove a taint (the trailing - removes it)
kubectl taint node <node> key=value:NoSchedule-

# Add a toleration to tolerate a taint
```

```yaml
# Pod spec with toleration
spec:
  tolerations:
    - key: "dedicated"
      operator: "Equal"
      value: "gpu"
      effect: "NoSchedule"
```

**Verify**: Pod's tolerations match all relevant node taints. Use `kubectl describe pod` to confirm scheduling.

### 5. Fix PVC binding issues

PVCs must bind to PVs before pods mounting them can schedule. [src3, src6]

```bash
# Check PVC status
kubectl get pvc -n <namespace>

# Describe PVC for events
kubectl describe pvc <pvc-name> -n <namespace>

# Check available PVs
kubectl get pv

# Check StorageClass
kubectl get storageclass

# Check if StorageClass has a working provisioner
kubectl describe storageclass <name>
```

```yaml
# Common fix: ensure StorageClass exists and matches
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: data-pvc
spec:
  accessModes:
    - ReadWriteOnce
  storageClassName: standard  # Must match an existing StorageClass
  resources:
    requests:
      storage: 10Gi
```

**Verify**: PVC shows "Bound" status. Pod events no longer mention "unbound PersistentVolumeClaims".

### 6. Fix nodeSelector and affinity mismatches

Pod selectors must match actual node labels. [src4, src7]

```bash
# Check pod's nodeSelector
kubectl get pod <pod> -o jsonpath='{.spec.nodeSelector}'

# Check pod's affinity rules
kubectl get pod <pod> -o jsonpath='{.spec.affinity}' | python -m json.tool

# List all node labels
kubectl get nodes --show-labels

# Add a label to a node
kubectl label node <node> disktype=ssd

# Remove a label
kubectl label node <node> disktype-
```

```yaml
# Example: nodeSelector that matches an existing label
spec:
  nodeSelector:
    disktype: ssd
    kubernetes.io/os: linux
```

**Verify**: At least one schedulable node has labels matching all selectors.

### 7. Check for ResourceQuota and node cordon

Namespace quotas and cordoned nodes block scheduling. [src1, src3]

```bash
# Check ResourceQuota in namespace
kubectl get resourcequota -n <namespace>
kubectl describe resourcequota -n <namespace>

# Check for cordoned (unschedulable) nodes
kubectl get nodes
# STATUS column: "Ready,SchedulingDisabled" = cordoned

# Uncordon a node
kubectl uncordon <node-name>

# Check LimitRange (default resource constraints)
kubectl get limitrange -n <namespace>
kubectl describe limitrange -n <namespace>
```

**Verify**: Quota has headroom; no nodes are unexpectedly cordoned.

## Code Examples

### Comprehensive Pending pod diagnostic script

> Full script: [comprehensive-pending-pod-diagnostic-script.sh](scripts/comprehensive-pending-pod-diagnostic-script.sh) (100 lines)

```bash
#!/bin/bash
# Input:  Pod name and namespace
# Output: Complete diagnostic report for Pending pods
POD="$1"
NS="${2:-default}"
# ... (see full script)
```

### Production-ready pod with proper scheduling config

> Full script: [production-ready-pod-with-proper-scheduling-config.yml](scripts/production-ready-pod-with-proper-scheduling-config.yml) (74 lines)

```yaml
# Input:  Pod that needs specific scheduling constraints
# Output: Deployment with nodeSelector, tolerations, affinity, and topology spread
apiVersion: apps/v1
kind: Deployment
metadata:
# ... (see full script)
```

### Cluster capacity audit script

> Full script: [cluster-capacity-audit-script.py](scripts/cluster-capacity-audit-script.py) (96 lines)

```python
#!/usr/bin/env python3
# Input:  Kubernetes cluster access via kubectl
# Output: Capacity report showing schedulable headroom per node
import subprocess
import json
# ... (see full script)
```

## Anti-Patterns

### Wrong: Requesting more resources than any node can provide

```yaml
# ❌ BAD — no 3-node cluster has a single node with 64Gi [src1, src4]
resources:
  requests:
    cpu: "16"
    memory: 64Gi
# Scheduler message: "0/3 nodes are available: 3 Insufficient memory"
```

### Correct: Size requests based on actual node capacity

```yaml
# ✅ GOOD — fits within typical node allocatable [src1, src4]
resources:
  requests:
    cpu: 500m
    memory: 512Mi
  limits:
    cpu: "1"
    memory: 1Gi
```

### Wrong: nodeSelector for labels that don't exist

```yaml
# ❌ BAD — no node has this label; pod will be Pending forever [src4, src7]
spec:
  nodeSelector:
    gpu-type: a100        # Label doesn't exist on any node!
    environment: staging   # This one too
```

### Correct: Verify labels exist before using nodeSelector

```bash
# ✅ GOOD — check labels first, then set selector [src4, src7]
kubectl get nodes --show-labels | grep gpu-type
# If label doesn't exist, add it:
kubectl label node worker-1 gpu-type=a100
```

```yaml
spec:
  nodeSelector:
    gpu-type: a100
```

### Wrong: Ignoring taints on all nodes

```yaml
# ❌ BAD — all nodes tainted with "dedicated=special:NoSchedule"
# but pod has no toleration. Stays Pending forever. [src2, src4]
spec:
  containers:
    - name: app
      image: myapp:1.0
  # No tolerations section!
```

### Correct: Add matching tolerations

```yaml
# ✅ GOOD — toleration matches the node taint [src2, src4]
spec:
  containers:
    - name: app
      image: myapp:1.0
  tolerations:
    - key: "dedicated"
      operator: "Equal"
      value: "special"
      effect: "NoSchedule"
```

### Wrong: RequiredDuringScheduling anti-affinity with too many replicas

```yaml
# ❌ BAD — 5 replicas with required anti-affinity on a 3-node cluster [src7]
spec:
  replicas: 5
  template:
    spec:
      affinity:
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            - labelSelector:
                matchLabels:
                  app: myapp
              topologyKey: kubernetes.io/hostname
# 2 pods will be Pending forever — only 3 nodes available
```

### Correct: Use preferredDuringScheduling for anti-affinity

```yaml
# ✅ GOOD — preferred anti-affinity allows co-location as fallback [src7]
spec:
  replicas: 5
  template:
    spec:
      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
            - weight: 100
              podAffinityTerm:
                labelSelector:
                  matchLabels:
                    app: myapp
                topologyKey: kubernetes.io/hostname
```

## Common Pitfalls

- **Requests vs limits confusion**: Scheduling uses *requests*, not limits. A pod requesting 4Gi is unschedulable on a node with 3Gi allocatable even if the pod would never actually use 4Gi. Right-size requests based on actual usage data from `kubectl top`. [src1, src4]
- **WaitForFirstConsumer PVCs**: PVCs with `volumeBindingMode: WaitForFirstConsumer` stay Pending until a pod needs them — this is by design, not a bug. The pod will also show Pending until the PVC binds. [src6]
- **Master/control-plane taints**: Control plane nodes have `node-role.kubernetes.io/control-plane:NoSchedule` by default. Add a toleration or use worker nodes. On single-node clusters (minikube, k3s), remove this taint. [src2, src4]
- **Hidden resource consumers**: DaemonSets, kube-system pods, and node allocations (~10% reserved for system) reduce schedulable capacity. Use `kubectl describe node` -> "Allocated resources" to see the true picture. [src4, src5]
- **Pod Anti-Affinity deadlocks**: RequiredDuringScheduling anti-affinity with more replicas than nodes creates an unsatisfiable constraint. Use PreferredDuringScheduling instead. [src7]
- **ResourceQuota silently blocks**: If namespace quota is exhausted, new pods stay Pending with a "exceeded quota" event. Easy to miss if you're not checking namespace-level events. [src1, src3]
- **maxUnavailable percentage rounding**: For deployments with fewer than 4 replicas, a 25% `maxUnavailable` rounds down to 0, which blocks rolling updates entirely. Use absolute values (e.g., `maxUnavailable: 1`) for small deployments. [src4]
- **Node conditions blocking scheduling**: Nodes reporting `DiskPressure: True`, `MemoryPressure: True`, or `Ready: False/Unknown` are excluded from scheduling even if they have allocatable resources. Check node conditions with `kubectl describe node`. [src4, src5]

## Diagnostic Commands

> Full script: [diagnostic-commands.sh](scripts/diagnostic-commands.sh) (29 lines)

```bash
# === Find Pending pods ===
kubectl get pods -A --field-selector=status.phase=Pending
# === Pod details (MOST IMPORTANT) ===
kubectl describe pod <pod> -n <ns>                    # Focus on Events section
kubectl get events -n <ns> --field-selector involvedObject.name=<pod>
# === Node resources ===
kubectl top nodes
kubectl describe nodes | grep -A5 "Allocatable:"
kubectl describe node <node> | grep -A20 "Allocated resources"
# === Taints ===
kubectl get nodes -o custom-columns=NAME:.metadata.name,TAINTS:.spec.taints[*].key
# === Labels ===
kubectl get nodes --show-labels
# === PVC ===
kubectl get pvc -n <ns>
kubectl get pv
kubectl get storageclass
# === Quotas ===
kubectl get resourcequota -n <ns>
kubectl describe limitrange -n <ns>
# === Schedulability ===
kubectl get nodes                    # SchedulingDisabled = cordoned
kubectl uncordon <node>
# === Node conditions ===
kubectl get nodes -o custom-columns=NAME:.metadata.name,READY:.status.conditions[-1:].status
kubectl describe node <node> | grep -A5 "Conditions:"
# === Cluster Autoscaler (managed K8s) ===
kubectl get events -n kube-system | grep cluster-autoscaler
```

## Version History & Compatibility

| Version | Behavior | Key Changes |
|---|---|---|
| K8s 1.36 (2026-04) | Current | Gang Scheduling Beta (PodGroup API); Workload-Aware Preemption alpha; topology-aware workload scheduling alpha (KEP-5732); DRA `firstAvailable` device requests; ResourceClaim sharing across PodGroup members [src9] |
| K8s 1.35 (2025-12) | Stable | In-Place Pod Resize GA; gang scheduling alpha; mutable PV node affinity; opportunistic batch scheduling beta [src8] |
| K8s 1.34 (2025-08) | Stable | Async scheduler API interactions; nominatedNodeName set for more pods; improved scheduler throughput [src1] |
| K8s 1.32 (2025-04) | Stable | QueueingHint beta (faster requeue of Pending pods) [src1] |
| K8s 1.31 (2024-12) | Stable | Image volume type; topology manager improvements [src1] |
| K8s 1.29 (2024) | Stable | Improved scheduling hints; sidecar containers GA [src1] |
| K8s 1.27 (2023) | Stable | In-place resource resize alpha (reduces Pending from resizing) [src8] |
| K8s 1.24 (2022) | Stable | Non-graceful node shutdown; improved PV topology [src6] |
| K8s 1.19 (2020) | Stable | PodTopologySpreadConstraints GA [src7] |
| K8s 1.18 (2020) | WaitForFirstConsumer default | More StorageClasses default to delayed binding [src6] |

## When to Use / When Not to Use

| Use When | Don't Use When | Use Instead |
|---|---|---|
| Pod status shows "Pending" | Pod shows CrashLoopBackOff | Debug container crash (exit code, logs) |
| Events mention scheduling failures | Pod shows ContainerCreating | Wait or debug image pull / volume mount |
| No node selected for the pod | Pod is Running but not Ready | Debug readiness probe |
| PVC is stuck in Pending | Pod was Evicted | Check node pressure conditions |
| Cluster Autoscaler not scaling up | Pod OOMKilled | Debug memory limits (not scheduling) |

## Important Caveats

- The scheduler considers resource *requests*, not actual usage. A node with 4Gi allocatable but 3.5Gi already requested has only 500Mi headroom for new pods — even if actual usage is only 1Gi. Use Vertical Pod Autoscaler (VPA) to right-size requests automatically. K8s 1.35 VPA InPlaceOrRecreate mode is now beta. [src1, src8]
- On managed Kubernetes (EKS, GKE, AKS), your node pool may have auto-scaling. A pending pod can trigger node scale-up, but it takes 1-5 minutes for the new node to be ready. Don't immediately reduce requests — wait for the autoscaler. Monitor `NotTriggerScaleUp` events from Cluster Autoscaler. [src4]
- `WaitForFirstConsumer` PVCs are zone-aware. They bind to a PV in the same zone as the node where the pod schedules. If your PV is in zone-a but the only available node is in zone-b, both the PVC and the pod stay Pending. [src6]
- Namespace-level `ResourceQuota` and `LimitRange` can silently inject defaults or block pods. Always check `kubectl describe quota -n <ns>` and `kubectl describe limitrange -n <ns>` when debugging Pending pods. [src1, src3]
- Pod priority and preemption: Higher-priority pods can preempt lower-priority ones. If your pod is stuck Pending and you see other pods being evicted, check `PriorityClass` objects in the cluster. In K8s 1.35, In-Place Pod Resize also respects priority for deferred resizes. [src1, src8]
- In multi-tenant clusters, each team's namespace may have its own quota. A team might have plenty of cluster capacity but their namespace quota is full. [src3]
- K8s 1.36+ gang scheduling (PodGroup, Beta) and Workload-Aware Preemption (alpha) change the unit of analysis: groups of related pods now schedule and preempt together. If you're on 1.36+ and a job's pods are stuck Pending without obvious resource issues, run `kubectl describe podgroup <name>` first — the group may be waiting for ALL members to fit before binding any. Workload-Aware Preemption also means an entire lower-priority group is preempted as one rather than picking off individual pods. [src9]

## Related Units

- [Kubernetes CrashLoopBackOff](/software/debugging/kubernetes-crashloopbackoff/2026)
- [Docker Container Won't Start](/software/debugging/docker-container-wont-start/2026)
- [Docker OOMKilled](/software/debugging/docker-oomkilled/2026)