--- # === IDENTITY === id: software/debugging/python-memory-leaks/2026 canonical_question: "How do I find and fix memory leaks in Python?" aliases: - "Python memory leak" - "Python memory usage growing" - "Python process memory increasing" - "Python tracemalloc" - "Python objgraph" - "Python gc collect" - "Python reference cycle" - "Python MemoryError" - "Python RSS growing" - "Python memray profiling" entity_type: software_reference domain: software > debugging > python_memory_leaks region: global jurisdiction: global temporal_scope: 2020-2026 # === VERIFICATION === last_verified: 2026-02-23 confidence: 0.93 version: 2.0 first_published: 2026-02-17 # === TEMPORAL VALIDITY === temporal_validity: status: stable last_breaking_change: "Python 3.13 free-threading experimental (2024-10); Python 3.14 free-threading supported (2025-10)" next_review: 2026-08-22 change_sensitivity: medium # === CONSTRAINTS === constraints: - "tracemalloc only tracks Python-level allocations — C extensions (numpy, pandas, Pillow) allocate via C malloc and are invisible to tracemalloc; use memray or RSS monitoring instead" - "In free-threaded Python 3.13t/3.14t builds, the GC uses a different implementation with stop-the-world pauses — gc.collect() behavior and timing differs from GIL-enabled builds" - "gc.collect() only collects reference cycles — objects with non-zero straight reference counts cannot be collected; you must remove all references manually" - "Never call gc.disable() in production without a replacement strategy — disabling GC causes all cyclic garbage to accumulate indefinitely" - "memray is Linux/macOS only (no native Windows support) — on Windows use tracemalloc, memory_profiler, or WSL" - "tracemalloc.start(N) with N>1 stores N frames per allocation — use N=1 in production (low overhead), N=10-25 for debugging (high overhead)" # === SKIP CONDITIONS === skip_this_unit_if: - condition: "Process is killed by OS/container OOM rather than leaking slowly" use_instead: "software/debugging/docker-oomkilled/2026" - condition: "Memory spike is during a single large operation (not gradual growth)" use_instead: "Use streaming/chunked processing — not a leak, just large data" - condition: "Issue is high CPU usage, not memory growth" use_instead: "software/debugging/python-cpu-profiling/2026" # === AGENT HINTS === inputs_needed: - key: python_version question: "Which Python version are you running?" type: choice options: ["3.8-3.12 (GIL-enabled)", "3.13+ (GIL-enabled)", "3.13t/3.14t (free-threaded)"] - key: app_type question: "What kind of application is leaking memory?" type: choice options: ["Web app (Django/Flask/FastAPI)", "Data pipeline/ETL", "Long-running script/daemon", "Jupyter notebook"] - key: growth_pattern question: "How does memory grow?" type: choice options: ["Slowly over hours/days", "Per-request in a web app", "Per-iteration in a loop", "Sudden spike on specific operation"] # === DISTRIBUTION === canonical_source: "https://knowledgelib.io/software/debugging/python-memory-leaks/2026" suggested_citation: "Source: knowledgelib.io — AI Knowledge Library (verified 2026-02-23)" # === RELATED UNITS === related_kos: related_to: - id: "software/debugging/nodejs-memory-leaks/2026" label: "Node.js Memory Leaks" - id: "software/debugging/python-importerror/2026" label: "Python ImportError/ModuleNotFoundError" solves: - id: "software/debugging/docker-oomkilled/2026" label: "Docker OOMKilled" often_confused_with: - id: "software/debugging/django-n-plus-1/2026" label: "Django N+1 Query Problems (slow, not leaking)" # === SOURCES (5-8 authoritative sources) === sources: - id: src1 title: "Python — tracemalloc: Trace memory allocations" author: Python Software Foundation url: https://docs.python.org/3/library/tracemalloc.html type: official_docs published: 2025-10-01 reliability: authoritative - id: src2 title: "Python — gc: Garbage Collector Interface" author: Python Software Foundation url: https://docs.python.org/3/library/gc.html type: official_docs published: 2025-10-01 reliability: authoritative - id: src3 title: "Python — weakref: Weak References" author: Python Software Foundation url: https://docs.python.org/3/library/weakref.html type: official_docs published: 2025-10-01 reliability: authoritative - id: src4 title: "Real Python — Python Memory Management and Tips" author: Real Python url: https://realpython.com/python-memory-management/ type: technical_blog published: 2025-01-01 reliability: high - id: src5 title: "Bloomberg Engineering — memory_profiler and tracemalloc for Production" author: Bloomberg Engineering url: https://www.bloomberg.com/company/stories/debugging-memory-leaks-python/ type: technical_blog published: 2024-06-01 reliability: high - id: src6 title: "objgraph — Python Object Graph Visualization" author: Marius Gedminas url: https://mg.pov.lt/objgraph/ type: official_docs published: 2024-01-01 reliability: high - id: src7 title: "memray — A Memory Profiler for Python" author: Bloomberg url: https://github.com/bloomberg/memray type: community_resource published: 2025-10-01 reliability: high - id: src8 title: "Quansight Labs — Free-Threaded GC Optimizations in Python 3.14" author: Quansight Labs url: https://labs.quansight.org/blog/free-threaded-gc-3-14 type: technical_blog published: 2025-06-01 reliability: high --- # How to Find and Fix Memory Leaks in Python ## TL;DR - **Bottom line**: Python uses reference counting + cyclic garbage collection. Memory leaks occur when objects remain referenced longer than needed. The 5 most common causes: (1) unbounded caches/lists growing forever, (2) reference cycles with `__del__`, (3) global/class-level mutable state, (4) unclosed resources (files, DB connections), (5) C extension memory not tracked by Python's GC. - **Key tool/command**: `tracemalloc` (built-in) — take two snapshots and compare: `tracemalloc.start(); snap1 = tracemalloc.take_snapshot(); ...code...; snap2 = tracemalloc.take_snapshot(); print(snap2.compare_to(snap1, 'lineno'))`. For production: `memray` (Bloomberg's profiler). - **Watch out for**: `functools.lru_cache` without `maxsize` — defaults to 128 entries but can still grow large. Unbounded `@lru_cache(maxsize=None)` is a common leak source. - **Works with**: Python 3.4+ (tracemalloc), Python 3.7+ (memray, Linux/macOS only). Applies to all applications: Django, Flask, FastAPI, data pipelines, long-running scripts. Free-threaded builds (3.13t/3.14t) have different GC behavior — see Constraints. ## Constraints - **tracemalloc is Python-only**: It does not track allocations made by C extensions (numpy, pandas, Pillow, etc.). If RSS grows but tracemalloc shows no growth, the leak is in native code — use memray or valgrind instead. [src1, src7] - **Free-threaded GC differs**: Python 3.13t/3.14t use a different garbage collector with stop-the-world pauses instead of incremental collection. `gc.collect()` behavior and timing differ from GIL-enabled builds. Test memory debugging workflows on the same build you deploy. [src2, src8] - **gc.collect() cannot fix straight references**: It only collects reference cycles. Objects with non-zero reference counts from non-cyclic references stay alive — you must `del` all references manually. [src2] - **Never disable GC without a plan**: `gc.disable()` causes all cyclic garbage to accumulate indefinitely. Only disable if you explicitly call `gc.collect()` at controlled intervals (e.g., between request batches). [src2] - **memray requires Linux/macOS**: No native Windows support. On Windows, use tracemalloc, memory_profiler, or run under WSL. [src7] - **tracemalloc frame depth trades off**: `tracemalloc.start(N)` stores N frames per allocation. N=1 for production (minimal overhead), N=10-25 for debugging (significant overhead and memory use). [src1] ## Quick Reference | # | Cause | Likelihood | Signature | Fix | |---|---|---|---|---| | 1 | Unbounded caches/lists/dicts | ~25% of cases | Memory grows linearly with requests/iterations | Add max size or TTL; use `lru_cache(maxsize=N)` [src4, src5] | | 2 | Reference cycles with `__del__` | ~15% of cases | Objects in `gc.garbage` list; GC can't collect | Remove `__del__`; use `weakref.finalize()` instead [src2, src3] | | 3 | Global/class-level mutable state | ~15% of cases | Memory grows across requests in web apps | Move state to instance level; clear between requests [src4, src5] | | 4 | Closures capturing large objects | ~10% of cases | `tracemalloc` shows function allocation growing | Capture only needed values, not entire objects [src4] | | 5 | Unclosed file handles/connections | ~8% of cases | OS file descriptor limit reached; RSS grows | Use `with` statement (context managers) [src4, src5] | | 6 | C extension / ctypes memory | ~7% of cases | `tracemalloc` shows low but `RSS` is high | Call extension's free/cleanup; use `memray` to trace native allocs [src1, src7] | | 7 | Event handlers / callbacks accumulating | ~6% of cases | Listener list grows; objects never GC'd | Use `weakref` for callbacks; remove listeners explicitly [src3] | | 8 | Thread-local storage not cleaned | ~5% of cases | Memory grows per thread in thread pools | Clear thread-locals; use `threading.local()` carefully [src4] | | 9 | Pandas/NumPy copies not freed | ~5% of cases | DataFrame copies accumulate in memory | Use `del df; gc.collect()`; avoid unnecessary copies [src4, src5] | | 10 | Logging handlers with large buffers | ~4% of cases | Log records accumulate in `MemoryHandler` | Set buffer limits; use `StreamHandler` for production [src5] | ## Decision Tree ``` START ├── Is memory growing slowly over time? │ ├── YES → Likely a leak in application code ↓ │ └── NO → Spike on specific operation? │ ├── YES → Profile that operation with tracemalloc [src1] │ └── NO → Check total dataset size vs available RAM ├── Does tracemalloc show the growth? │ ├── YES → tracemalloc snapshot comparison shows the allocation site [src1] │ │ ├── Is it a cache/dict/list growing? → Add max size or TTL [src4] │ │ ├── Is it closures? → Refactor to capture less [src4] │ │ └── Is it in a library? → Check for known issues, update library │ └── NO → Memory growth is outside Python's allocator │ ├── C extension? → Use memray or valgrind to trace native allocations [src7] │ └── Check RSS vs Python heap: python -c "import resource; print(resource.getrusage(resource.RUSAGE_SELF).ru_maxrss)" ├── Are there objects in gc.garbage? │ ├── YES → Reference cycles with __del__. Remove __del__ or use weakref.finalize [src2, src3] │ └── NO ↓ ├── Use objgraph to find what's holding references │ └── objgraph.show_backrefs(obj) shows the reference chain [src6] ├── Is it a web application (Django/Flask/FastAPI)? │ ├── YES → Check middleware, global state, request-scoped caching [src4] │ └── NO → Check long-running loop, accumulating data structures └── Using free-threaded Python (3.13t/3.14t)? ├── YES → GC uses stop-the-world collector; check for increased memory overhead (~15-20%) [src8] └── NO → Standard GC applies ``` ## Step-by-Step Guide ### 1. Confirm there's actually a memory leak Not all memory growth is a leak — Python's memory allocator may hold freed memory for reuse. [src4] ```python import os import psutil def get_memory_mb(): """Get current process memory in MB.""" process = psutil.Process(os.getpid()) return process.memory_info().rss / 1024 / 1024 print(f"Memory: {get_memory_mb():.1f} MB") ``` ```bash # Monitor from outside watch -n 1 "ps -p $(pgrep -f your_script) -o rss,vsz,pid,comm" # Or use /proc on Linux cat /proc/$(pgrep -f your_script)/status | grep VmRSS ``` **Verify**: Run your application for several minutes/hours. If RSS grows continuously without plateauing, you have a leak. ### 2. Use tracemalloc to find the allocation source `tracemalloc` is built into Python and shows exactly which lines allocate the most memory. [src1] ```python import tracemalloc # Start tracing (do this as early as possible) tracemalloc.start() # Take a baseline snapshot snapshot1 = tracemalloc.take_snapshot() # ... run your code / handle some requests ... # Take a second snapshot snapshot2 = tracemalloc.take_snapshot() # Compare: what grew? top_stats = snapshot2.compare_to(snapshot1, 'lineno') print("\n=== Top 10 memory increases ===") for stat in top_stats[:10]: print(stat) # Group by filename for a higher-level view top_files = snapshot2.compare_to(snapshot1, 'filename') print("\n=== Top 5 files by growth ===") for stat in top_files[:5]: print(stat) ``` **Verify**: The output shows file:line → size increase, pointing you directly to the leaking code. ### 3. Use objgraph to trace reference chains When you know *what* is leaking but not *why* it's kept alive, `objgraph` shows the reference chain. [src6] ```python import objgraph # Show most common object types in memory objgraph.show_most_common_types(limit=15) # Show growth between two points objgraph.show_growth(limit=10) # Call once # ... do some work ... objgraph.show_growth(limit=10) # Shows what increased # Find reference chain keeping an object alive # Useful for: "Why isn't this object being GC'd?" leaking_objects = objgraph.by_type('MyClassName') if leaking_objects: objgraph.show_backrefs( leaking_objects[-1], max_depth=5, filename='refs.png' # Generates a graph image ) ``` **Verify**: The reference graph shows the chain from the leaking object back to a root reference (global, module, class attribute). ### 4. Check for reference cycles Python's GC handles most cycles, but `__del__` methods can prevent collection. [src2] ```python import gc # Enable GC debugging gc.set_debug(gc.DEBUG_SAVEALL) # Force collection collected = gc.collect() print(f"Collected {collected} objects") # Check for uncollectable objects (cycles with __del__) if gc.garbage: print(f"WARNING: {len(gc.garbage)} uncollectable objects!") for obj in gc.garbage[:5]: print(f" Type: {type(obj)}, repr: {repr(obj)[:100]}") ``` **Verify**: If `gc.garbage` is non-empty, you have `__del__` methods creating uncollectable cycles. ### 5. Fix unbounded caches The most common cause: data structures that grow without limit. [src4, src5] ```python from functools import lru_cache from collections import OrderedDict import time # WRONG — unbounded cache grows forever cache = {} def get_user(user_id): if user_id not in cache: cache[user_id] = db.fetch(user_id) return cache[user_id] # FIX 1 — Use lru_cache with maxsize @lru_cache(maxsize=1000) def get_user(user_id): return db.fetch(user_id) # FIX 2 — TTL cache (time-based eviction) class TTLCache: def __init__(self, maxsize=1000, ttl=300): self._cache = OrderedDict() self._maxsize = maxsize self._ttl = ttl def get(self, key): if key in self._cache: value, timestamp = self._cache[key] if time.time() - timestamp < self._ttl: self._cache.move_to_end(key) return value del self._cache[key] return None def set(self, key, value): self._cache[key] = (value, time.time()) if len(self._cache) > self._maxsize: self._cache.popitem(last=False) ``` **Verify**: Memory plateaus after reaching cache maxsize. ### 6. Use weakref to break reference chains When you need references that don't prevent garbage collection. [src3] ```python import weakref # WRONG — observer list prevents GC of observers class EventBus: def __init__(self): self._listeners = [] # Strong references def subscribe(self, callback): self._listeners.append(callback) # CORRECT — weak references allow GC class EventBus: def __init__(self): self._listeners = [] def subscribe(self, callback): ref = weakref.ref(callback, self._remove_dead) self._listeners.append(ref) def _remove_dead(self, ref): self._listeners = [r for r in self._listeners if r() is not None] def emit(self, *args): for ref in self._listeners[:]: callback = ref() if callback is not None: callback(*args) # For caches: WeakValueDictionary cache = weakref.WeakValueDictionary() ``` **Verify**: Objects are collected when no other strong references exist. ### 7. Profile in production with memray For production systems where overhead matters. Linux/macOS only. [src7] ```bash # Install memray pip install memray # Run your script under memray python -m memray run -o output.bin your_script.py # Generate flame graph python -m memray flamegraph output.bin -o flamegraph.html # Generate summary python -m memray summary output.bin # Live tracking (attach to running process) python -m memray attach PID # For web apps: track specific endpoints python -m memray run --live your_app.py ``` **Verify**: Flame graph shows allocation hotspots with file names and line numbers. ## Code Examples ### Memory leak detector for long-running applications > Full script: [memory-leak-detector-for-long-running-applications.py](scripts/memory-leak-detector-for-long-running-applications.py) (44 lines) ```python # Input: Long-running service with gradually increasing memory # Output: Automatic leak detection with periodic snapshots import tracemalloc import time import threading # ... (see full script) ``` ### Django middleware for per-request memory tracking > Full script: [django-middleware-for-per-request-memory-tracking.py](scripts/django-middleware-for-per-request-memory-tracking.py) (30 lines) ```python # Input: Django app with memory growing across requests # Output: Middleware that logs memory usage per request import tracemalloc import logging import psutil # ... (see full script) ``` ### Automatic memory cleanup utility > Full script: [automatic-memory-cleanup-utility.py](scripts/automatic-memory-cleanup-utility.py) (30 lines) ```python # Input: Data pipeline processing large files in a loop # Output: Context manager that ensures cleanup after each batch import gc import tracemalloc import contextlib # ... (see full script) ``` ## Anti-Patterns ### Wrong: Unbounded cache with no eviction ```python # BAD — dictionary grows forever as more users are fetched [src4, src5] _user_cache = {} def get_user(user_id): if user_id not in _user_cache: _user_cache[user_id] = db.query(f"SELECT * FROM users WHERE id = {user_id}") return _user_cache[user_id] ``` ### Correct: Bounded cache with maxsize ```python # GOOD — LRU eviction prevents unbounded growth [src4, src5] from functools import lru_cache @lru_cache(maxsize=500) def get_user(user_id): return db.query(f"SELECT * FROM users WHERE id = %s", (user_id,)) # Clear when needed: get_user.cache_clear() ``` ### Wrong: Using __del__ with circular references ```python # BAD — __del__ prevents GC from collecting the cycle [src2, src3] class Node: def __init__(self, parent=None): self.parent = parent self.children = [] if parent: parent.children.append(self) def __del__(self): print(f"Deleting {self}") # GC can't determine safe deletion order ``` ### Correct: Use weakref.finalize or context manager ```python # GOOD — weakref.finalize runs cleanup without blocking GC [src2, src3] import weakref class Node: def __init__(self, parent=None): self.parent = weakref.ref(parent) if parent else None self.children = [] if parent: parent.children.append(self) weakref.finalize(self, Node._cleanup, id(self)) @staticmethod def _cleanup(node_id): print(f"Cleaned up node {node_id}") ``` ### Wrong: Not closing resources in long-running processes ```python # BAD — file handles accumulate, eventually hitting OS limits [src4] def process_files(file_list): results = [] for path in file_list: f = open(path) # Never closed! results.append(f.read()) return results ``` ### Correct: Use context managers ```python # GOOD — resources automatically cleaned up [src4] def process_files(file_list): results = [] for path in file_list: with open(path) as f: results.append(f.read()) return results ``` ## Common Pitfalls - **`@lru_cache(maxsize=None)` is unbounded**: This creates an infinite cache that never evicts. Use `maxsize=N` for a bounded LRU cache, or regularly call `cache_clear()`. [src4] - **`del` doesn't free memory immediately**: `del` just decrements the reference count. If other references exist, the object stays alive. Use `gc.collect()` after `del` for cycles. [src2] - **`__del__` prevents cycle collection**: If objects in a reference cycle have `__del__`, CPython can't determine safe deletion order and puts them in `gc.garbage` instead of collecting them. Use `weakref.finalize()`. [src2, src3] - **Pandas `.copy()` vs view confusion**: Some Pandas operations return views (no extra memory) and some return copies (double memory). Use `df.copy()` explicitly when needed, `del df` when done. [src5] - **`tracemalloc` overhead**: `tracemalloc.start(N)` stores N frames per allocation. Higher N = more useful traces but more overhead. Use 1 for production, 10-25 for debugging. [src1] - **RSS does not equal Python heap**: Resident Set Size includes memory from C extensions, shared libraries, and the OS. `tracemalloc` only tracks Python allocations. If RSS grows but `tracemalloc` doesn't show growth, suspect C extensions. [src1, src7] - **Free-threaded Python uses more memory**: Python 3.13t/3.14t free-threaded builds use approximately 15-20% more memory than GIL-enabled builds due to per-thread overhead and the different GC implementation. Factor this into baseline measurements. [src8] - **Django `DEBUG=True` stores all queries**: `django.db.connection.queries` grows indefinitely in long-running processes when `DEBUG=True`. Always set `DEBUG=False` in production. [src4] ## Diagnostic Commands > Full script: [diagnostic-commands.sh](scripts/diagnostic-commands.sh) (27 lines) ```bash # Monitor process memory (Linux) watch -n 1 "ps -p PID -o rss,vsz,pid,comm" # Detailed memory map (Linux) pmap -x PID | tail -5 # Python: check current memory usage # ... (see full script) ``` ## Version History & Compatibility | Version | Behavior | Key Changes | |---|---|---| | Python 3.14 (Oct 2025) | Free-threading supported | PEP 779 promotes free-threading to supported status; new stop-the-world GC for free-threaded builds; ~5-10% single-thread overhead in free-threaded mode [src2, src8] | | Python 3.13 (Oct 2024) | Free-threading experimental | Experimental `--disable-gil` build (PEP 703); immortal objects expanded; `gc.collect()` gains generation parameter improvements [src2, src8] | | Python 3.12 | Improved GC | Immortal objects (PEP 683); reduced GC overhead for long-lived objects; per-interpreter GIL (PEP 684) [src2] | | Python 3.11 | Zero-cost exceptions | `tracemalloc` improvements; faster `gc.collect()` [src1, src2] | | Python 3.10 | Stable | `gc.freeze()` improved for fork safety [src2] | | Python 3.9 | Stable | `gc.get_objects(generation)` to inspect specific generations [src2] | | Python 3.7 | memray support | `memray` requires 3.7+; `tracemalloc` mature [src1, src7] | | Python 3.4 | `tracemalloc` introduced | Built-in memory allocation tracing [src1] | | Python 3.3 | GC improvements | Better handling of `__del__` in cycles (PEP 442) [src2] | ## When to Use / When Not to Use | Use When | Don't Use When | Use Instead | |---|---|---| | RSS / heap grows continuously over time | Memory spikes during a known large operation | Optimize the operation or use streaming | | Application eventually crashes with `MemoryError` | Total data doesn't fit in RAM | Use chunked processing, databases, or disk | | Memory doesn't decrease after GC | Process uses expected memory for its dataset | Normal behavior — not a leak | | Web app memory grows across requests | Memory drops when requests finish | Normal per-request allocation | | Container gets OOMKilled repeatedly | Container memory limit is too low for workload | Increase container memory limit | ## Important Caveats - Python's memory allocator (`pymalloc`) doesn't return freed memory to the OS in small chunks. After freeing many small objects, RSS may not decrease even though the memory is available for reuse by Python. This is normal, not a leak. - `gc.collect()` only collects reference cycles. Objects with non-zero reference counts from straight references (no cycle) can't be collected — you must remove all references manually. - In multiprocessing (`fork`), child processes share memory pages with the parent until they write (copy-on-write). This means memory usage *appears* doubled but isn't actually until modifications occur. - `numpy` and `pandas` allocate memory outside Python's tracked heap (via C). `tracemalloc` won't show their allocations. Use `memray` or check RSS directly. - Django's `DEBUG = True` stores all SQL queries in `django.db.connection.queries`, which grows indefinitely in long-running processes. Always set `DEBUG = False` in production. - Free-threaded Python 3.13t/3.14t builds use a fundamentally different garbage collector. Memory profiling results from GIL-enabled builds may not be representative of free-threaded behavior. Always profile on the same build you deploy to production. [src8] ## Related Units - [Node.js Memory Leaks](/software/debugging/nodejs-memory-leaks/2026) - [Python ImportError/ModuleNotFoundError](/software/debugging/python-importerror/2026) - [Docker OOMKilled](/software/debugging/docker-oomkilled/2026) - [Django N+1 Query Problems](/software/debugging/django-n-plus-1/2026)