How to Find and Fix Memory Leaks in Python

How do I find and fix memory leaks in Python?

TL;DR

Constraints

Quick Reference

# Cause Likelihood Signature Fix
1 Unbounded caches/lists/dicts ~25% of cases Memory grows linearly with requests/iterations Add max size or TTL; use lru_cache(maxsize=N) [src4, src5]
2 Reference cycles with __del__ ~15% of cases Objects in gc.garbage list; GC can't collect Remove __del__; use weakref.finalize() instead [src2, src3]
3 Global/class-level mutable state ~15% of cases Memory grows across requests in web apps Move state to instance level; clear between requests [src4, src5]
4 Closures capturing large objects ~10% of cases tracemalloc shows function allocation growing Capture only needed values, not entire objects [src4]
5 Unclosed file handles/connections ~8% of cases OS file descriptor limit reached; RSS grows Use with statement (context managers) [src4, src5]
6 C extension / ctypes memory ~7% of cases tracemalloc shows low but RSS is high Call extension's free/cleanup; use memray to trace native allocs [src1, src7]
7 Event handlers / callbacks accumulating ~6% of cases Listener list grows; objects never GC'd Use weakref for callbacks; remove listeners explicitly [src3]
8 Thread-local storage not cleaned ~5% of cases Memory grows per thread in thread pools Clear thread-locals; use threading.local() carefully [src4]
9 Pandas/NumPy copies not freed ~5% of cases DataFrame copies accumulate in memory Use del df; gc.collect(); avoid unnecessary copies [src4, src5]
10 Logging handlers with large buffers ~4% of cases Log records accumulate in MemoryHandler Set buffer limits; use StreamHandler for production [src5]

Decision Tree

START
├── Is memory growing slowly over time?
│   ├── YES → Likely a leak in application code ↓
│   └── NO → Spike on specific operation?
│       ├── YES → Profile that operation with tracemalloc [src1]
│       └── NO → Check total dataset size vs available RAM
├── Does tracemalloc show the growth?
│   ├── YES → tracemalloc snapshot comparison shows the allocation site [src1]
│   │   ├── Is it a cache/dict/list growing? → Add max size or TTL [src4]
│   │   ├── Is it closures? → Refactor to capture less [src4]
│   │   └── Is it in a library? → Check for known issues, update library
│   └── NO → Memory growth is outside Python's allocator
│       ├── C extension? → Use memray or valgrind [src7]
│       └── Check RSS vs Python heap
├── Are there objects in gc.garbage?
│   ├── YES → Reference cycles with __del__. Remove __del__ or use weakref.finalize [src2, src3]
│   └── NO ↓
├── Use objgraph to find what's holding references
│   └── objgraph.show_backrefs(obj) shows the reference chain [src6]
├── Is it a web application (Django/Flask/FastAPI)?
│   ├── YES → Check middleware, global state, request-scoped caching [src4]
│   └── NO → Check long-running loop, accumulating data structures
└── Using free-threaded Python (3.13t/3.14t)?
    ├── YES → GC uses stop-the-world collector; expect ~15-20% more memory [src8]
    └── NO → Standard GC applies

Step-by-Step Guide

1. Confirm there's actually a memory leak

Not all memory growth is a leak — Python's memory allocator may hold freed memory for reuse. [src4]

import os, psutil

def get_memory_mb():
    process = psutil.Process(os.getpid())
    return process.memory_info().rss / 1024 / 1024

print(f"Memory: {get_memory_mb():.1f} MB")

Verify: Run your application for several minutes/hours. If RSS grows continuously without plateauing, you have a leak.

2. Use tracemalloc to find the allocation source

tracemalloc is built into Python and shows exactly which lines allocate the most memory. [src1]

import tracemalloc

tracemalloc.start()
snapshot1 = tracemalloc.take_snapshot()

# ... run your code ...

snapshot2 = tracemalloc.take_snapshot()
top_stats = snapshot2.compare_to(snapshot1, 'lineno')
print("\n=== Top 10 memory increases ===")
for stat in top_stats[:10]:
    print(stat)

Verify: The output shows file:line → size increase, pointing you directly to the leaking code.

3. Use objgraph to trace reference chains

When you know what is leaking but not why it's kept alive, objgraph shows the reference chain. [src6]

import objgraph

objgraph.show_most_common_types(limit=15)
objgraph.show_growth(limit=10)

leaking_objects = objgraph.by_type('MyClassName')
if leaking_objects:
    objgraph.show_backrefs(leaking_objects[-1], max_depth=5, filename='refs.png')

Verify: The reference graph shows the chain from the leaking object back to a root reference.

4. Check for reference cycles

Python's GC handles most cycles, but __del__ methods can prevent collection. [src2]

import gc

gc.set_debug(gc.DEBUG_SAVEALL)
collected = gc.collect()
print(f"Collected {collected} objects")

if gc.garbage:
    print(f"WARNING: {len(gc.garbage)} uncollectable objects!")
    for obj in gc.garbage[:5]:
        print(f"  Type: {type(obj)}, repr: {repr(obj)[:100]}")

Verify: If gc.garbage is non-empty, you have __del__ methods creating uncollectable cycles.

5. Fix unbounded caches

The most common cause: data structures that grow without limit. [src4, src5]

from functools import lru_cache

# FIX — Use lru_cache with maxsize
@lru_cache(maxsize=1000)
def get_user(user_id):
    return db.fetch(user_id)

Verify: Memory plateaus after reaching cache maxsize.

6. Use weakref to break reference chains

When you need references that don't prevent garbage collection. [src3]

import weakref

class EventBus:
    def __init__(self):
        self._listeners = []

    def subscribe(self, callback):
        ref = weakref.ref(callback, self._remove_dead)
        self._listeners.append(ref)

    def _remove_dead(self, ref):
        self._listeners = [r for r in self._listeners if r() is not None]

    def emit(self, *args):
        for ref in self._listeners[:]:
            callback = ref()
            if callback is not None:
                callback(*args)

Verify: Objects are collected when no other strong references exist.

7. Profile in production with memray

For production systems where overhead matters. Linux/macOS only. [src7]

# Install and run
pip install memray
python -m memray run -o output.bin your_script.py
python -m memray flamegraph output.bin -o flamegraph.html
python -m memray summary output.bin
# Attach to running process
python -m memray attach PID

Verify: Flame graph shows allocation hotspots with file names and line numbers.

Code Examples

Memory leak detector for long-running applications

# Input:  Long-running service with gradually increasing memory
# Output: Automatic leak detection with periodic snapshots

import tracemalloc, time, threading, logging

logger = logging.getLogger(__name__)

class MemoryLeakDetector:
    def __init__(self, interval=60, top_n=5, growth_threshold_mb=10):
        self.interval = interval
        self.top_n = top_n
        self.threshold = growth_threshold_mb * 1024 * 1024
        self._baseline = None
        self._running = False

    def start(self):
        tracemalloc.start(25)
        self._baseline = tracemalloc.take_snapshot()
        self._running = True
        thread = threading.Thread(target=self._monitor, daemon=True)
        thread.start()
        logger.info("Memory leak detector started")

    def _monitor(self):
        while self._running:
            time.sleep(self.interval)
            snapshot = tracemalloc.take_snapshot()
            stats = snapshot.compare_to(self._baseline, 'lineno')
            total_growth = sum(s.size_diff for s in stats if s.size_diff > 0)
            if total_growth > self.threshold:
                logger.warning(
                    f"Memory grew by {total_growth / 1024 / 1024:.1f} MB since baseline!"
                )
                for stat in stats[:self.top_n]:
                    logger.warning(f"  {stat}")

    def stop(self):
        self._running = False
        tracemalloc.stop()

# Usage
detector = MemoryLeakDetector(interval=30, growth_threshold_mb=50)
detector.start()

Django middleware for per-request memory tracking

# Input:  Django app with memory growing across requests
# Output: Middleware that logs memory usage per request

import tracemalloc, logging, psutil, os

logger = logging.getLogger('memory')

class MemoryTrackingMiddleware:
    def __init__(self, get_response):
        self.get_response = get_response
        self.request_count = 0
        tracemalloc.start()

    def __call__(self, request):
        self.request_count += 1
        rss_before = psutil.Process(os.getpid()).memory_info().rss
        snapshot_before = tracemalloc.take_snapshot()

        response = self.get_response(request)

        rss_after = psutil.Process(os.getpid()).memory_info().rss
        rss_diff = (rss_after - rss_before) / 1024 / 1024

        if self.request_count % 100 == 0 or rss_diff > 5:
            snapshot_after = tracemalloc.take_snapshot()
            stats = snapshot_after.compare_to(snapshot_before, 'lineno')
            top = '; '.join(str(s) for s in stats[:3])
            logger.info(
                f"[Req #{self.request_count}] {request.path} "
                f"RSS: {rss_after/1024/1024:.0f}MB (delta {rss_diff:+.1f}MB) Top: {top}"
            )
        return response

Automatic memory cleanup utility

# Input:  Data pipeline processing large files in a loop
# Output: Context manager that ensures cleanup after each batch

import gc, tracemalloc, contextlib

@contextlib.contextmanager
def memory_guard(label="operation", warn_mb=100):
    """Context manager that tracks and cleans up memory."""
    gc.collect()
    if tracemalloc.is_tracing():
        snap_before = tracemalloc.take_snapshot()
    try:
        yield
    finally:
        gc.collect()
        if tracemalloc.is_tracing():
            snap_after = tracemalloc.take_snapshot()
            stats = snap_after.compare_to(snap_before, 'filename')
            growth = sum(s.size_diff for s in stats if s.size_diff > 0)
            if growth > warn_mb * 1024 * 1024:
                print(f"WARNING: [{label}] Memory grew by {growth/1024/1024:.1f} MB")
                for s in stats[:3]:
                    print(f"  {s}")

# Usage
tracemalloc.start()
for batch_file in glob.glob("data/*.csv"):
    with memory_guard(f"Processing {batch_file}", warn_mb=50):
        df = pd.read_csv(batch_file)
        process(df)
        del df

Anti-Patterns

Wrong: Unbounded cache with no eviction

# BAD — dictionary grows forever [src4, src5]
_user_cache = {}

def get_user(user_id):
    if user_id not in _user_cache:
        _user_cache[user_id] = db.query(f"SELECT * FROM users WHERE id = {user_id}")
    return _user_cache[user_id]

Correct: Bounded cache with maxsize

# GOOD — LRU eviction prevents unbounded growth [src4, src5]
from functools import lru_cache

@lru_cache(maxsize=500)
def get_user(user_id):
    return db.query(f"SELECT * FROM users WHERE id = %s", (user_id,))
# Clear when needed: get_user.cache_clear()

Wrong: Using __del__ with circular references

# BAD — __del__ prevents GC from collecting the cycle [src2, src3]
class Node:
    def __init__(self, parent=None):
        self.parent = parent
        self.children = []
        if parent:
            parent.children.append(self)

    def __del__(self):
        print(f"Deleting {self}")

Correct: Use weakref.finalize or context manager

# GOOD — weakref breaks the cycle, finalize runs cleanup [src2, src3]
import weakref

class Node:
    def __init__(self, parent=None):
        self.parent = weakref.ref(parent) if parent else None
        self.children = []
        if parent:
            parent.children.append(self)
        weakref.finalize(self, Node._cleanup, id(self))

    @staticmethod
    def _cleanup(node_id):
        print(f"Cleaned up node {node_id}")

Wrong: Not closing resources in long-running processes

# BAD — file handles accumulate, eventually hitting OS limits [src4]
def process_files(file_list):
    results = []
    for path in file_list:
        f = open(path)  # Never closed!
        results.append(f.read())
    return results

Correct: Use context managers

# GOOD — resources automatically cleaned up [src4]
def process_files(file_list):
    results = []
    for path in file_list:
        with open(path) as f:
            results.append(f.read())
    return results

Common Pitfalls

Diagnostic Commands

# Monitor process memory (Linux)
watch -n 1 "ps -p PID -o rss,vsz,pid,comm"

# Python: check current memory usage
python -c "
import tracemalloc, gc
tracemalloc.start()
gc.collect()
snap = tracemalloc.take_snapshot()
for stat in snap.statistics('filename')[:10]:
    print(stat)
"

# Check for uncollectable objects
python -c "import gc; gc.set_debug(gc.DEBUG_SAVEALL); gc.collect(); print(f'Garbage: {len(gc.garbage)} objects')"

# Install profiling tools
pip install objgraph psutil memory-profiler memray

# memray: profile a script
python -m memray run -o output.bin script.py
python -m memray flamegraph output.bin

# memory_profiler: line-by-line
python -m memory_profiler script.py

# objgraph: show common types
python -c "import objgraph; objgraph.show_most_common_types(limit=20)"

# Check gc stats
python -c "import gc; print(gc.get_stats())"

Version History & Compatibility

Version Behavior Key Changes
Python 3.14 (Oct 2025) Free-threading supported PEP 779 promotes free-threading to supported status; new stop-the-world GC; ~5-10% single-thread overhead in free-threaded mode [src2, src8]
Python 3.13 (Oct 2024) Free-threading experimental Experimental --disable-gil build (PEP 703); immortal objects expanded [src2, src8]
Python 3.12 Improved GC Immortal objects (PEP 683); reduced GC overhead for long-lived objects; per-interpreter GIL (PEP 684) [src2]
Python 3.11 Zero-cost exceptions tracemalloc improvements; faster gc.collect() [src1, src2]
Python 3.10 Stable gc.freeze() improved for fork safety [src2]
Python 3.9 Stable gc.get_objects(generation) to inspect specific generations [src2]
Python 3.7 memray support memray requires 3.7+; tracemalloc mature [src1, src7]
Python 3.4 tracemalloc introduced Built-in memory allocation tracing [src1]
Python 3.3 GC improvements Better handling of __del__ in cycles (PEP 442) [src2]

When to Use / When Not to Use

Use When Don't Use When Use Instead
RSS / heap grows continuously Memory spikes during a known large operation Optimize the operation or use streaming
Application crashes with MemoryError Total data doesn't fit in RAM Use chunked processing, databases, or disk
Memory doesn't decrease after GC Process uses expected memory for its dataset Normal behavior — not a leak
Web app memory grows across requests Memory drops when requests finish Normal per-request allocation
Container gets OOMKilled repeatedly Container memory limit is too low for workload Increase container memory limit

Important Caveats