--- # === IDENTITY === id: software/patterns/retry-exponential-backoff/2026 canonical_question: "How do I implement retry with exponential backoff and jitter?" aliases: - "exponential backoff algorithm" - "retry with jitter implementation" - "backoff and retry pattern" - "how to retry API calls with exponential backoff" - "thundering herd retry prevention" - "decorrelated jitter vs full jitter" - "retry strategy for distributed systems" - "exponential backoff formula" entity_type: software_reference domain: software > patterns > retry exponential backoff region: global jurisdiction: global temporal_scope: 2015-2026 # === VERIFICATION === last_verified: 2026-02-24 confidence: 0.92 version: 1.0 first_published: 2026-02-24 # === TEMPORAL VALIDITY === temporal_validity: status: stable last_breaking_change: null next_review: 2026-08-23 change_sensitivity: low # === CONSTRAINTS === constraints: - "NEVER retry non-idempotent operations (POST creating resources, financial transactions) without an idempotency key" - "ALWAYS set a maximum retry count (3-5 attempts) -- infinite retries cause cascading failures" - "ALWAYS add jitter to backoff delays -- pure exponential backoff without jitter causes thundering herd" - "NEVER retry 4xx client errors (400, 401, 403, 404) -- these are not transient and will never succeed" - "Cap maximum delay (typically 30-60 seconds) -- unbounded exponential growth wastes time without benefit" # === SKIP CONDITIONS === skip_this_unit_if: - condition: "Need to stop calling a failing service entirely after repeated failures" use_instead: "Circuit breaker pattern" - condition: "Need to handle partial failures in a multi-step workflow" use_instead: "Saga pattern / compensating transactions" - condition: "Need request hedging (send parallel requests, take first response)" use_instead: "Hedged requests / speculative execution pattern" # === AGENT HINTS === inputs_needed: - key: retry_type question: "What backoff strategy do you need?" type: choice options: ["fixed", "linear", "exponential", "decorrelated jitter"] - key: max_retries question: "What is the maximum number of retry attempts?" type: text default: "3" - key: idempotent_operation question: "Is the operation being retried idempotent (safe to repeat)?" type: boolean # === DISTRIBUTION === canonical_source: "https://knowledgelib.io/software/patterns/retry-exponential-backoff/2026" suggested_citation: "Source: knowledgelib.io -- AI Knowledge Library (verified 2026-02-24)" # === RELATED UNITS === related_kos: depends_on: [] related_to: - id: software/patterns/circuit-breaker/2026 label: "Circuit Breaker Pattern" - id: software/patterns/idempotency-patterns/2026 label: "Idempotency Patterns" - id: software/patterns/connection-pooling/2026 label: "Connection Pooling" solves: - id: software/patterns/webhook-implementation/2026 label: "Webhook Implementation" alternative_to: [] often_confused_with: - id: software/patterns/circuit-breaker/2026 label: "Circuit Breaker Pattern" # === SOURCES (7 authoritative sources) === sources: - id: src1 title: "Exponential Backoff And Jitter" author: AWS Architecture Blog url: https://aws.amazon.com/blogs/architecture/exponential-backoff-and-jitter/ type: technical_blog published: 2015-03-04 reliability: authoritative - id: src2 title: "Timeouts, retries, and backoff with jitter" author: Amazon Builders' Library url: https://aws.amazon.com/builders-library/timeouts-retries-and-backoff-with-jitter/ type: official_docs published: 2019-09-01 reliability: authoritative - id: src3 title: "Retry strategy - Cloud Storage" author: Google Cloud url: https://docs.google.com/storage/docs/retry-strategy type: official_docs published: 2024-01-15 reliability: high - id: src4 title: "Retry resilience strategy" author: Polly (.NET) url: https://www.pollydocs.org/strategies/retry.html type: official_docs published: 2024-06-01 reliability: high - id: src5 title: "Tenacity - Retrying library for Python" author: Julien Danjou url: https://tenacity.readthedocs.io/ type: official_docs published: 2024-08-01 reliability: high - id: src6 title: "Retry with backoff pattern" author: AWS Prescriptive Guidance url: https://docs.aws.amazon.com/prescriptive-guidance/latest/cloud-design-patterns/retry-backoff.html type: official_docs published: 2024-03-01 reliability: high - id: src7 title: "Retry behavior - AWS SDKs and Tools" author: AWS url: https://docs.aws.amazon.com/sdkref/latest/guide/feature-retry-behavior.html type: official_docs published: 2024-10-01 reliability: high --- # Retry with Exponential Backoff and Jitter ## TL;DR - **Bottom line**: Exponential backoff with full jitter prevents thundering herd problems by spreading retry attempts across time -- use `delay = min(cap, base * 2^attempt) * random()` for optimal load distribution on failing services. - **Key tool/command**: `delay = min(cap, base * 2^attempt) * random(0, 1)` (full jitter formula) - **Watch out for**: Retrying non-idempotent operations without an idempotency key -- this causes duplicate writes, double charges, and data corruption. - **Works with**: Any language, any protocol. Used by AWS SDKs (default), Google Cloud client libraries, gRPC, Polly (.NET), Tenacity (Python), and all major HTTP clients. ## Constraints - NEVER retry non-idempotent operations (POST creating resources, financial transactions) without an idempotency key. [src2] - ALWAYS set a maximum retry count (3-5 attempts) -- infinite retries cause cascading failures and resource exhaustion. [src6] - ALWAYS add jitter to backoff delays -- pure exponential backoff without jitter causes synchronized retry storms (thundering herd). [src1] - NEVER retry 4xx client errors (400, 401, 403, 404) -- these are permanent failures that will never succeed on retry. Only retry 429 (rate limit) and 5xx (server error). [src3] - Cap maximum delay at 30-60 seconds -- unbounded exponential growth (2^20 = 1M seconds) wastes time without improving success rate. [src7] ## Quick Reference | Strategy | Formula | Thundering Herd Risk | Fairness | Complexity | Best For | |---|---|---|---|---|---| | No backoff (fixed) | `delay = constant` | Very High | Equal | Trivial | Never use for retries | | Linear backoff | `delay = base * attempt` | High | Equal | Low | Simple rate limiting | | Exponential (no jitter) | `delay = min(cap, base * 2^attempt)` | High | Equal | Low | Prototype only | | Full jitter | `delay = random(0, min(cap, base * 2^attempt))` | Very Low | High | Low | Default recommendation | | Equal jitter | `delay = min(cap, base * 2^attempt)/2 + random(0, min(cap, base * 2^attempt)/2)` | Low | Medium | Low | Predictable minimum wait | | Decorrelated jitter | `delay = min(cap, random(base, prev_delay * 3))` | Low | Medium | Medium | Stateful clients | | Fixed delay | `delay = constant` | Very High | Equal | Trivial | Polling, not retries | | Exponential + token bucket | Full jitter + token bucket rate limit | Very Low | High | Medium | AWS SDK default | ## Decision Tree ``` START: Is the failed operation retryable? | +-- Is the error transient (429, 408, 500, 502, 503, 504, network timeout)? | +-- NO (400, 401, 403, 404, 422) --> Do NOT retry. Return error immediately. | +-- YES ↓ | +-- Is the operation idempotent (safe to repeat)? | +-- NO --> Add idempotency key or do NOT retry. | +-- YES ↓ | +-- How many concurrent clients may retry simultaneously? | +-- Few (<10) --> Exponential backoff (jitter optional) | +-- Many (10-1000) --> Full jitter (recommended default) | +-- Very many (>1000) --> Full jitter + token bucket + circuit breaker | +-- Do you need a guaranteed minimum wait time? | +-- YES --> Equal jitter (half fixed, half random) | +-- NO --> Full jitter (lowest total load) | +-- Is this a long-running background job? +-- YES --> Decorrelated jitter (independent of attempt count) +-- NO --> Full jitter with max 3-5 attempts ``` ## Step-by-Step Guide ### 1. Identify retryable errors Only retry transient failures. Server errors (5xx) and rate limits (429) are retryable. Client errors (4xx except 429, 408) are permanent and must not be retried. [src3] ```python RETRYABLE_STATUS_CODES = {408, 429, 500, 502, 503, 504} RETRYABLE_EXCEPTIONS = (ConnectionError, TimeoutError, OSError) def is_retryable(error): if isinstance(error, RETRYABLE_EXCEPTIONS): return True if hasattr(error, 'status_code'): return error.status_code in RETRYABLE_STATUS_CODES return False ``` **Verify**: `is_retryable(HTTPError(status_code=503))` returns `True`; `is_retryable(HTTPError(status_code=400))` returns `False`. ### 2. Implement the full jitter formula Full jitter provides the best load distribution across retrying clients. The formula randomizes the delay between 0 and the exponential ceiling. [src1] ```python import random def full_jitter_delay(attempt, base=1.0, cap=30.0): """Calculate retry delay with full jitter. Args: attempt: Zero-based attempt number (0 = first retry) base: Base delay in seconds (default 1s) cap: Maximum delay cap in seconds (default 30s) Returns: Delay in seconds (float) """ exp_delay = min(cap, base * (2 ** attempt)) return random.uniform(0, exp_delay) ``` **Verify**: `full_jitter_delay(0)` returns value in [0, 1.0]; `full_jitter_delay(5)` returns value in [0, 30.0]. ### 3. Build the retry loop with maximum attempts Wrap the retryable operation in a loop with configurable max attempts, applying the jitter delay between each attempt. [src2] ```python import time import logging def retry_with_backoff(fn, max_attempts=4, base=1.0, cap=30.0): """Execute fn with retry and exponential backoff + full jitter. Args: fn: Callable that may raise retryable exceptions max_attempts: Total attempts including the initial call base: Base delay in seconds cap: Maximum delay cap Returns: Result of fn() Raises: Last exception if all attempts fail """ last_exception = None for attempt in range(max_attempts): try: return fn() except Exception as e: last_exception = e if not is_retryable(e) or attempt == max_attempts - 1: raise delay = full_jitter_delay(attempt, base, cap) logging.warning( f"Attempt {attempt + 1}/{max_attempts} failed: {e}. " f"Retrying in {delay:.2f}s" ) time.sleep(delay) raise last_exception ``` **Verify**: Function retries on 503, gives up on 400, raises after max_attempts exhausted. ### 4. Add retry budget / token bucket (for high-scale systems) Prevent retry amplification by limiting the total retry rate across all requests. AWS SDKs use a token bucket: 500 initial tokens, 5 tokens per successful call refunded, 5 tokens consumed per retry. [src7] ```python import threading class RetryBudget: """Token bucket to limit global retry rate.""" def __init__(self, max_tokens=500, refill_per_success=5, cost_per_retry=5): self.tokens = max_tokens self.max_tokens = max_tokens self.refill = refill_per_success self.cost = cost_per_retry self._lock = threading.Lock() def acquire(self): with self._lock: if self.tokens >= self.cost: self.tokens -= self.cost return True return False # Budget exhausted -- do not retry def success(self): with self._lock: self.tokens = min(self.max_tokens, self.tokens + self.refill) ``` **Verify**: After 100 consecutive failures (100 * 5 = 500 tokens consumed), `acquire()` returns `False`. ### 5. Respect Retry-After headers When the server sends a `Retry-After` header (common with 429 and 503), use the server-specified delay instead of your calculated backoff. [src3] ```python def get_retry_delay(response, attempt, base=1.0, cap=30.0): """Use Retry-After header if present, otherwise calculate backoff.""" retry_after = response.headers.get('Retry-After') if retry_after: try: return min(float(retry_after), cap) except ValueError: pass # Non-numeric Retry-After (HTTP-date format) return full_jitter_delay(attempt, base, cap) ``` **Verify**: Response with `Retry-After: 5` returns `5.0`; without header falls back to jitter calculation. ## Code Examples ### Python (tenacity): Decorator-Based Retry ```python # Input: Any function that may raise transient errors # Output: Automatic retry with exponential backoff + jitter from tenacity import ( retry, stop_after_attempt, wait_exponential_jitter, retry_if_exception_type, before_sleep_log ) import logging import httpx # pip install httpx>=0.27 logger = logging.getLogger(__name__) @retry( stop=stop_after_attempt(4), wait=wait_exponential_jitter(initial=1, max=30, jitter=5), retry=retry_if_exception_type((httpx.TransportError, httpx.HTTPStatusError)), before_sleep=before_sleep_log(logger, logging.WARNING), reraise=True, ) def fetch_with_retry(url: str) -> dict: """Fetch URL with automatic retry on transient errors.""" response = httpx.get(url, timeout=10) if response.status_code in (429, 500, 502, 503, 504): response.raise_for_status() return response.json() ``` ### Node.js: Async Retry with Full Jitter ```javascript // Input: Async function that may throw retryable errors // Output: Result of successful call, or throws after max attempts async function retryWithBackoff(fn, { maxAttempts = 4, baseDelay = 1000, // milliseconds maxDelay = 30000, } = {}) { for (let attempt = 0; attempt < maxAttempts; attempt++) { try { return await fn(); } catch (error) { const isRetryable = error.status >= 500 || error.status === 429 || error.code === 'ECONNRESET' || error.code === 'ETIMEDOUT'; if (!isRetryable || attempt === maxAttempts - 1) throw error; // Full jitter: random delay between 0 and exponential ceiling const expDelay = Math.min(maxDelay, baseDelay * 2 ** attempt); const delay = Math.random() * expDelay; console.warn(`Attempt ${attempt + 1}/${maxAttempts} failed. Retrying in ${delay.toFixed(0)}ms`); await new Promise(r => setTimeout(r, delay)); } } } // Usage: // const data = await retryWithBackoff(() => fetch('https://api.example.com/data')); ``` ### Go: Retry with Context and Full Jitter > Full script: [go-retry-with-context-and-full-jitter.go](scripts/go-retry-with-context-and-full-jitter.go) (39 lines) ```go // Input: Context, retryable function // Output: Result of successful call, or error after max attempts package retry import ( "context" # ... (see full script) ``` ### Java: Retry with Exponential Backoff > Full script: [java-retry-with-exponential-backoff.java](scripts/java-retry-with-exponential-backoff.java) (32 lines) ```java // Input: Callable that may throw retryable exceptions // Output: Result of successful call, or throws after max attempts import java.util.Set; import java.util.concurrent.ThreadLocalRandom; public class RetryWithBackoff { # ... (see full script) ``` ## Anti-Patterns ### Wrong: Retry without any backoff ```python # BAD -- hammering a failing service makes the outage worse for attempt in range(5): try: return call_api() except Exception: pass # Retry immediately with zero delay ``` ### Correct: Retry with exponential backoff and jitter ```python # GOOD -- spreading retries over time lets the service recover for attempt in range(5): try: return call_api() except TransientError: delay = min(30, 1.0 * 2 ** attempt) * random.random() time.sleep(delay) ``` ### Wrong: Retrying non-retryable errors (400, 401, 404) ```python # BAD -- 400 Bad Request will fail every time, wasting 3 retry attempts @retry(stop=stop_after_attempt(4), wait=wait_exponential()) def create_user(data): response = httpx.post('/users', json=data) response.raise_for_status() # Retries even on 400/401/404! ``` ### Correct: Only retry transient errors ```python # GOOD -- only retry server errors and rate limits @retry( stop=stop_after_attempt(4), wait=wait_exponential_jitter(initial=1, max=30), retry=retry_if_exception(lambda e: getattr(e, 'response', None) and e.response.status_code in (429, 500, 502, 503, 504)), ) def create_user(data): response = httpx.post('/users', json=data) response.raise_for_status() ``` ### Wrong: Exponential backoff without jitter ```python # BAD -- all 1000 clients retry at exactly 1s, 2s, 4s, 8s -> thundering herd delay = base * (2 ** attempt) time.sleep(delay) ``` ### Correct: Always add full jitter ```python # GOOD -- clients spread retries uniformly, preventing synchronized storms delay = random.uniform(0, min(cap, base * (2 ** attempt))) time.sleep(delay) ``` ### Wrong: Infinite retries with no maximum ```python # BAD -- retries forever, consuming threads/connections/memory while True: try: return call_api() except Exception: time.sleep(2 ** attempt) attempt += 1 # Grows to 2^100+ seconds ``` ### Correct: Bounded retries with a cap ```python # GOOD -- give up after max_attempts and let the caller handle the failure MAX_ATTEMPTS = 4 CAP = 30.0 # seconds for attempt in range(MAX_ATTEMPTS): try: return call_api() except TransientError: if attempt == MAX_ATTEMPTS - 1: raise # Propagate after final attempt delay = random.uniform(0, min(CAP, 1.0 * 2 ** attempt)) time.sleep(delay) ``` ## Common Pitfalls - **No jitter causes thundering herd**: 1000 clients failing at the same time all retry at 1s, 2s, 4s, 8s -- hitting the server in synchronized waves. Fix: `delay = random.uniform(0, min(cap, base * 2^attempt))`. [src1] - **Retrying non-idempotent operations**: Retrying a POST /charge endpoint without an idempotency key creates duplicate charges. Fix: Pass `Idempotency-Key` header; server deduplicates based on key. [src2] - **Ignoring Retry-After headers**: When a server sends `Retry-After: 60`, ignoring it and retrying in 2s triggers rate limiting or bans. Fix: Parse `Retry-After` header and use server-specified delay as minimum. [src3] - **Retrying at every layer**: If client retries 3x, load balancer retries 3x, and gateway retries 3x, one user request generates 27 backend calls. Fix: Retry only at the outermost layer, or use a shared retry budget. [src2] - **Unbounded exponential delay**: Without a cap, `2^20 = 1,048,576` seconds (~12 days). Fix: `min(cap, base * 2^attempt)` with cap of 30-60 seconds. [src7] - **Not logging retry attempts**: Silent retries mask transient issues until they become chronic. Fix: Log every retry with attempt number, delay, and error. [src6] - **Retrying on circuit breaker open**: Continuing to retry when the circuit breaker is open wastes resources. Fix: Check circuit state before attempting retry. [src2] ## When to Use / When Not to Use | Use When | Don't Use When | Use Instead | |---|---|---| | Transient network errors (timeouts, DNS failures) | Client-side validation errors (4xx) | Return error immediately | | Rate-limited APIs returning 429 | Non-idempotent operations without idempotency keys | Idempotency pattern first, then retry | | Cloud service temporary unavailability (503) | Real-time user-facing requests with tight latency SLAs | Hedged requests / speculative execution | | Batch processing / background jobs | Service is consistently down (not transient) | Circuit breaker to fail fast | | Database connection pool exhaustion | Retries at multiple layers (client + LB + gateway) | Single retry point with retry budget | | Message queue consumer failures | Authentication/authorization errors (401, 403) | Re-authenticate, do not retry | ## Important Caveats - Full jitter has the lowest total server load according to AWS simulations, but takes slightly longer to complete than equal jitter -- choose based on whether you optimize for server health (full jitter) or client latency (equal jitter). [src1] - AWS SDKs default to 3 max attempts with 20-second max backoff. Google Cloud client libraries always enable jitter. Polly (.NET) v8+ uses `BackoffType.Exponential` with `UseJitter = true`. These defaults are well-tuned -- prefer library implementations over hand-rolled retries. [src7] - Retry amplification is the most dangerous failure mode: N clients each retrying M times generates N*M load on an already failing service. Use a global retry budget (token bucket) for services handling >100 requests/second. [src2] - When using gRPC, leverage built-in retry policies in the service config rather than implementing client-side retry. gRPC retry policies support exponential backoff with jitter natively. - In serverless environments (AWS Lambda, Cloud Functions), remember that retries consume invocation time and cost. Set aggressive max_attempts (2-3) for cost-sensitive workloads. ## Related Units - [Circuit Breaker Pattern](/software/patterns/circuit-breaker/2026) - [Idempotency Patterns](/software/patterns/idempotency-patterns/2026) - [Connection Pooling](/software/patterns/connection-pooling/2026) - [Webhook Implementation](/software/patterns/webhook-implementation/2026)