How to Find and Fix Memory Leaks in Python

TL;DR

• Bottom line: Python uses reference counting + cyclic garbage collection. Memory leaks occur when objects remain referenced longer than needed. The 5 most common causes: (1) unbounded caches/lists growing forever, (2) reference cycles with __del__, (3) global/class-level mutable state, (4) unclosed resources (files, DB connections), (5) C extension memory not tracked by Python's GC.
• Key tool/command: tracemalloc (built-in) — take two snapshots and compare: tracemalloc.start(); snap1 = tracemalloc.take_snapshot(); ...code...; snap2 = tracemalloc.take_snapshot(); print(snap2.compare_to(snap1, 'lineno')). For production: memray (Bloomberg's profiler).
• Watch out for: functools.lru_cache without maxsize — defaults to 128 entries but can still grow large. Unbounded @lru_cache(maxsize=None) is a common leak source.
• Works with: Python 3.4+ (tracemalloc), Python 3.7+ (memray, Linux/macOS only). Applies to all applications: Django, Flask, FastAPI, data pipelines, long-running scripts. Free-threaded builds (3.13t/3.14t) have different GC behavior — see Constraints.

Constraints

• tracemalloc is Python-only: It does not track allocations made by C extensions (numpy, pandas, Pillow, etc.). If RSS grows but tracemalloc shows no growth, the leak is in native code — use memray or valgrind instead. [src1, src7]
• Free-threaded GC differs: Python 3.13t/3.14t use a different garbage collector with stop-the-world pauses instead of incremental collection. gc.collect() behavior and timing differ from GIL-enabled builds. Test memory debugging workflows on the same build you deploy. [src2, src8]
• gc.collect() cannot fix straight references: It only collects reference cycles. Objects with non-zero reference counts from non-cyclic references stay alive — you must del all references manually. [src2]
• Never disable GC without a plan: gc.disable() causes all cyclic garbage to accumulate indefinitely. Only disable if you explicitly call gc.collect() at controlled intervals. [src2]
• memray requires Linux/macOS: No native Windows support. On Windows, use tracemalloc, memory_profiler, or run under WSL. [src7]
• tracemalloc frame depth trades off: tracemalloc.start(N) stores N frames per allocation. N=1 for production (minimal overhead), N=10-25 for debugging (significant overhead and memory use). [src1]

Quick Reference

Decision Tree

START
├── Is memory growing slowly over time?
│   ├── YES → Likely a leak in application code ↓
│   └── NO → Spike on specific operation?
│       ├── YES → Profile that operation with tracemalloc [src1]
│       └── NO → Check total dataset size vs available RAM
├── Does tracemalloc show the growth?
│   ├── YES → tracemalloc snapshot comparison shows the allocation site [src1]
│   │   ├── Is it a cache/dict/list growing? → Add max size or TTL [src4]
│   │   ├── Is it closures? → Refactor to capture less [src4]
│   │   └── Is it in a library? → Check for known issues, update library
│   └── NO → Memory growth is outside Python's allocator
│       ├── C extension? → Use memray or valgrind [src7]
│       └── Check RSS vs Python heap
├── Are there objects in gc.garbage?
│   ├── YES → Reference cycles with __del__. Remove __del__ or use weakref.finalize [src2, src3]
│   └── NO ↓
├── Use objgraph to find what's holding references
│   └── objgraph.show_backrefs(obj) shows the reference chain [src6]
├── Is it a web application (Django/Flask/FastAPI)?
│   ├── YES → Check middleware, global state, request-scoped caching [src4]
│   └── NO → Check long-running loop, accumulating data structures
└── Using free-threaded Python (3.13t/3.14t)?
    ├── YES → GC uses stop-the-world collector; expect ~15-20% more memory [src8]
    └── NO → Standard GC applies

Step-by-Step Guide

1. Confirm there's actually a memory leak

Not all memory growth is a leak — Python's memory allocator may hold freed memory for reuse. [src4]

import os, psutil

def get_memory_mb():
    process = psutil.Process(os.getpid())
    return process.memory_info().rss / 1024 / 1024

print(f"Memory: {get_memory_mb():.1f} MB")

Verify: Run your application for several minutes/hours. If RSS grows continuously without plateauing, you have a leak.

2. Use tracemalloc to find the allocation source

tracemalloc is built into Python and shows exactly which lines allocate the most memory. [src1]

import tracemalloc

tracemalloc.start()
snapshot1 = tracemalloc.take_snapshot()

# ... run your code ...

snapshot2 = tracemalloc.take_snapshot()
top_stats = snapshot2.compare_to(snapshot1, 'lineno')
print("\n=== Top 10 memory increases ===")
for stat in top_stats[:10]:
    print(stat)

Verify: The output shows file:line → size increase, pointing you directly to the leaking code.

3. Use objgraph to trace reference chains

When you know what is leaking but not why it's kept alive, objgraph shows the reference chain. [src6]

import objgraph

objgraph.show_most_common_types(limit=15)
objgraph.show_growth(limit=10)

leaking_objects = objgraph.by_type('MyClassName')
if leaking_objects:
    objgraph.show_backrefs(leaking_objects[-1], max_depth=5, filename='refs.png')

Verify: The reference graph shows the chain from the leaking object back to a root reference.

4. Check for reference cycles

Python's GC handles most cycles, but __del__ methods can prevent collection. [src2]

import gc

gc.set_debug(gc.DEBUG_SAVEALL)
collected = gc.collect()
print(f"Collected {collected} objects")

if gc.garbage:
    print(f"WARNING: {len(gc.garbage)} uncollectable objects!")
    for obj in gc.garbage[:5]:
        print(f"  Type: {type(obj)}, repr: {repr(obj)[:100]}")

Verify: If gc.garbage is non-empty, you have __del__ methods creating uncollectable cycles.

5. Fix unbounded caches

The most common cause: data structures that grow without limit. [src4, src5]

from functools import lru_cache

# FIX — Use lru_cache with maxsize
@lru_cache(maxsize=1000)
def get_user(user_id):
    return db.fetch(user_id)

Verify: Memory plateaus after reaching cache maxsize.

6. Use weakref to break reference chains

When you need references that don't prevent garbage collection. [src3]

import weakref

class EventBus:
    def __init__(self):
        self._listeners = []

    def subscribe(self, callback):
        ref = weakref.ref(callback, self._remove_dead)
        self._listeners.append(ref)

    def _remove_dead(self, ref):
        self._listeners = [r for r in self._listeners if r() is not None]

    def emit(self, *args):
        for ref in self._listeners[:]:
            callback = ref()
            if callback is not None:
                callback(*args)

Verify: Objects are collected when no other strong references exist.

7. Profile in production with memray

For production systems where overhead matters. Linux/macOS only. [src7]

# Install and run
pip install memray
python -m memray run -o output.bin your_script.py
python -m memray flamegraph output.bin -o flamegraph.html
python -m memray summary output.bin
# Attach to running process
python -m memray attach PID

Verify: Flame graph shows allocation hotspots with file names and line numbers.

Code Examples

Memory leak detector for long-running applications

# Input:  Long-running service with gradually increasing memory
# Output: Automatic leak detection with periodic snapshots

import tracemalloc, time, threading, logging

logger = logging.getLogger(__name__)

class MemoryLeakDetector:
    def __init__(self, interval=60, top_n=5, growth_threshold_mb=10):
        self.interval = interval
        self.top_n = top_n
        self.threshold = growth_threshold_mb * 1024 * 1024
        self._baseline = None
        self._running = False

    def start(self):
        tracemalloc.start(25)
        self._baseline = tracemalloc.take_snapshot()
        self._running = True
        thread = threading.Thread(target=self._monitor, daemon=True)
        thread.start()
        logger.info("Memory leak detector started")

    def _monitor(self):
        while self._running:
            time.sleep(self.interval)
            snapshot = tracemalloc.take_snapshot()
            stats = snapshot.compare_to(self._baseline, 'lineno')
            total_growth = sum(s.size_diff for s in stats if s.size_diff > 0)
            if total_growth > self.threshold:
                logger.warning(
                    f"Memory grew by {total_growth / 1024 / 1024:.1f} MB since baseline!"
                )
                for stat in stats[:self.top_n]:
                    logger.warning(f"  {stat}")

    def stop(self):
        self._running = False
        tracemalloc.stop()

# Usage
detector = MemoryLeakDetector(interval=30, growth_threshold_mb=50)
detector.start()

Django middleware for per-request memory tracking

# Input:  Django app with memory growing across requests
# Output: Middleware that logs memory usage per request

import tracemalloc, logging, psutil, os

logger = logging.getLogger('memory')

class MemoryTrackingMiddleware:
    def __init__(self, get_response):
        self.get_response = get_response
        self.request_count = 0
        tracemalloc.start()

    def __call__(self, request):
        self.request_count += 1
        rss_before = psutil.Process(os.getpid()).memory_info().rss
        snapshot_before = tracemalloc.take_snapshot()

        response = self.get_response(request)

        rss_after = psutil.Process(os.getpid()).memory_info().rss
        rss_diff = (rss_after - rss_before) / 1024 / 1024

        if self.request_count % 100 == 0 or rss_diff > 5:
            snapshot_after = tracemalloc.take_snapshot()
            stats = snapshot_after.compare_to(snapshot_before, 'lineno')
            top = '; '.join(str(s) for s in stats[:3])
            logger.info(
                f"[Req #{self.request_count}] {request.path} "
                f"RSS: {rss_after/1024/1024:.0f}MB (delta {rss_diff:+.1f}MB) Top: {top}"
            )
        return response

Automatic memory cleanup utility

# Input:  Data pipeline processing large files in a loop
# Output: Context manager that ensures cleanup after each batch

import gc, tracemalloc, contextlib

@contextlib.contextmanager
def memory_guard(label="operation", warn_mb=100):
    """Context manager that tracks and cleans up memory."""
    gc.collect()
    if tracemalloc.is_tracing():
        snap_before = tracemalloc.take_snapshot()
    try:
        yield
    finally:
        gc.collect()
        if tracemalloc.is_tracing():
            snap_after = tracemalloc.take_snapshot()
            stats = snap_after.compare_to(snap_before, 'filename')
            growth = sum(s.size_diff for s in stats if s.size_diff > 0)
            if growth > warn_mb * 1024 * 1024:
                print(f"WARNING: [{label}] Memory grew by {growth/1024/1024:.1f} MB")
                for s in stats[:3]:
                    print(f"  {s}")

# Usage
tracemalloc.start()
for batch_file in glob.glob("data/*.csv"):
    with memory_guard(f"Processing {batch_file}", warn_mb=50):
        df = pd.read_csv(batch_file)
        process(df)
        del df

Anti-Patterns

Wrong: Unbounded cache with no eviction

# BAD — dictionary grows forever [src4, src5]
_user_cache = {}

def get_user(user_id):
    if user_id not in _user_cache:
        _user_cache[user_id] = db.query(f"SELECT * FROM users WHERE id = {user_id}")
    return _user_cache[user_id]

Correct: Bounded cache with maxsize

# GOOD — LRU eviction prevents unbounded growth [src4, src5]
from functools import lru_cache

@lru_cache(maxsize=500)
def get_user(user_id):
    return db.query(f"SELECT * FROM users WHERE id = %s", (user_id,))
# Clear when needed: get_user.cache_clear()

Wrong: Using __del__ with circular references

# BAD — __del__ prevents GC from collecting the cycle [src2, src3]
class Node:
    def __init__(self, parent=None):
        self.parent = parent
        self.children = []
        if parent:
            parent.children.append(self)

    def __del__(self):
        print(f"Deleting {self}")

Correct: Use weakref.finalize or context manager

# GOOD — weakref breaks the cycle, finalize runs cleanup [src2, src3]
import weakref

class Node:
    def __init__(self, parent=None):
        self.parent = weakref.ref(parent) if parent else None
        self.children = []
        if parent:
            parent.children.append(self)
        weakref.finalize(self, Node._cleanup, id(self))

    @staticmethod
    def _cleanup(node_id):
        print(f"Cleaned up node {node_id}")

Wrong: Not closing resources in long-running processes

# BAD — file handles accumulate, eventually hitting OS limits [src4]
def process_files(file_list):
    results = []
    for path in file_list:
        f = open(path)  # Never closed!
        results.append(f.read())
    return results

Correct: Use context managers

# GOOD — resources automatically cleaned up [src4]
def process_files(file_list):
    results = []
    for path in file_list:
        with open(path) as f:
            results.append(f.read())
    return results

Common Pitfalls

• @lru_cache(maxsize=None) is unbounded: This creates an infinite cache that never evicts. Use maxsize=N for a bounded LRU cache, or regularly call cache_clear(). [src4]
• del doesn't free memory immediately: del just decrements the reference count. If other references exist, the object stays alive. Use gc.collect() after del for cycles. [src2]
• __del__ prevents cycle collection: If objects in a reference cycle have __del__, CPython can't determine safe deletion order and puts them in gc.garbage. Use weakref.finalize(). [src2, src3]
• Pandas .copy() vs view confusion: Some Pandas operations return views (no extra memory) and some return copies (double memory). Use df.copy() explicitly when needed, del df when done. [src5]
• tracemalloc overhead: tracemalloc.start(N) stores N frames per allocation. Higher N = more useful traces but more overhead. Use 1 for production, 10-25 for debugging. [src1]
• RSS does not equal Python heap: Resident Set Size includes memory from C extensions, shared libraries, and the OS. tracemalloc only tracks Python allocations. If RSS grows but tracemalloc doesn't, suspect C extensions. [src1, src7]
• Free-threaded Python uses more memory: Python 3.13t/3.14t free-threaded builds use approximately 15-20% more memory than GIL-enabled builds due to per-thread overhead and the different GC implementation. Factor this into baseline measurements. [src8]
• Django DEBUG=True stores all queries: django.db.connection.queries grows indefinitely in long-running processes when DEBUG=True. Always set DEBUG=False in production. [src4]

Diagnostic Commands

# Monitor process memory (Linux)
watch -n 1 "ps -p PID -o rss,vsz,pid,comm"

# Python: check current memory usage
python -c "
import tracemalloc, gc
tracemalloc.start()
gc.collect()
snap = tracemalloc.take_snapshot()
for stat in snap.statistics('filename')[:10]:
    print(stat)
"

# Check for uncollectable objects
python -c "import gc; gc.set_debug(gc.DEBUG_SAVEALL); gc.collect(); print(f'Garbage: {len(gc.garbage)} objects')"

# Install profiling tools
pip install objgraph psutil memory-profiler memray

# memray: profile a script
python -m memray run -o output.bin script.py
python -m memray flamegraph output.bin

# memory_profiler: line-by-line
python -m memory_profiler script.py

# objgraph: show common types
python -c "import objgraph; objgraph.show_most_common_types(limit=20)"

# Check gc stats
python -c "import gc; print(gc.get_stats())"

Version History & Compatibility

When to Use / When Not to Use

Important Caveats

• Python's memory allocator (pymalloc) doesn't return freed memory to the OS in small chunks. After freeing many small objects, RSS may not decrease even though the memory is available for reuse by Python. This is normal, not a leak.
• gc.collect() only collects reference cycles. Objects with non-zero reference counts from straight references (no cycle) can't be collected — you must remove all references manually.
• In multiprocessing (fork), child processes share memory pages with the parent until they write (copy-on-write). Memory usage appears doubled but isn't actually until modifications occur.
• numpy and pandas allocate memory outside Python's tracked heap (via C). tracemalloc won't show their allocations. Use memray or check RSS directly.
• Django's DEBUG = True stores all SQL queries in django.db.connection.queries, which grows indefinitely in long-running processes. Always set DEBUG = False in production.
• Free-threaded Python 3.13t/3.14t builds use a fundamentally different garbage collector. Memory profiling results from GIL-enabled builds may not be representative of free-threaded behavior. Always profile on the same build you deploy to production. [src8]