--- # === IDENTITY === id: software/debugging/graphql-n-plus-1-dataloader/2026 canonical_question: "How do I fix N+1 queries in GraphQL with DataLoader?" aliases: - "GraphQL N+1 problem DataLoader batching" - "How to batch GraphQL resolver queries" - "DataLoader per-request caching GraphQL" - "Reduce database calls in GraphQL nested resolvers" entity_type: software_reference domain: software > debugging > graphql_n_plus_1_dataloader region: global jurisdiction: global temporal_scope: 2020-2026 # === VERIFICATION === last_verified: 2026-02-23 confidence: 0.93 version: 1.0 first_published: 2026-02-23 # === TEMPORAL VALIDITY === temporal_validity: status: stable last_breaking_change: "DataLoader v2.0.0 (2020) — dropped Node <10 support" next_review: 2026-08-22 change_sensitivity: low # === CONSTRAINTS === constraints: - "DataLoader instances MUST be created per-request — a shared instance leaks cached data between users" - "The batch function MUST return an array the same length as the input keys array, in the same order" - "DataLoader requires an async/Promise-based environment (Node.js event loop or Python asyncio)" - "DataLoader caches by reference equality by default — use cacheKeyFn for complex key objects" # === SKIP CONDITIONS === skip_this_unit_if: - condition: "Query is slow but not due to repeated resolver calls (e.g., single expensive JOIN)" use_instead: "Database query optimization / indexing guide" - condition: "Using REST API, not GraphQL" use_instead: "REST API N+1 solutions (eager loading, includes parameter)" - condition: "N+1 occurs in ORM layer outside GraphQL (e.g., Django ORM select_related)" use_instead: "ORM-specific eager loading guide (select_related, prefetch_related, includes)" # === AGENT HINTS === inputs_needed: - key: "language" question: "Which language/framework are you using for your GraphQL server?" type: choice options: ["JavaScript/TypeScript (Apollo Server)", "JavaScript/TypeScript (graphql-js)", "Python (Strawberry)", "Python (Graphene)", "Ruby (graphql-ruby)", "Go (gqlgen)", "Java (graphql-java)", "Other"] - key: "architecture" question: "Is your GraphQL server monolithic or federated (Apollo Federation)?" type: choice options: ["Monolithic", "Federated (Apollo Federation)", "Schema stitching", "Not sure"] # === DISTRIBUTION === canonical_source: "https://knowledgelib.io/software/debugging/graphql-n-plus-1-dataloader/2026" suggested_citation: "Source: knowledgelib.io — AI Knowledge Library (verified 2026-02-23)" # === RELATED UNITS === related_kos: depends_on: [] related_to: [] solves: [] alternative_to: [] often_confused_with: [] # === SOURCES (7 authoritative sources) === sources: - id: src1 title: "Solving the N+1 Problem with DataLoader" author: GraphQL.js url: https://www.graphql-js.org/docs/n1-dataloader/ type: official_docs published: 2024-01-15 reliability: high - id: src2 title: "DataLoader — Generic utility for batching and caching" author: GraphQL Foundation (Facebook) url: https://github.com/graphql/dataloader type: official_docs published: 2024-06-10 reliability: authoritative - id: src3 title: "Handling the N+1 Problem" author: Apollo GraphQL url: https://www.apollographql.com/docs/graphos/schema-design/guides/handling-n-plus-one type: official_docs published: 2025-03-20 reliability: high - id: src4 title: "DataLoaders Guide" author: Strawberry GraphQL url: https://strawberry.rocks/docs/guides/dataloaders type: official_docs published: 2025-01-10 reliability: high - id: src5 title: "Graphene-Python DataLoader Documentation" author: Graphene-Python url: https://docs.graphene-python.org/en/latest/execution/dataloader/ type: official_docs published: 2024-08-15 reliability: high - id: src6 title: "Solving the N+1 Problem for GraphQL through Batching" author: Shopify Engineering url: https://shopify.engineering/solving-the-n-1-problem-for-graphql-through-batching type: technical_blog published: 2023-09-12 reliability: moderate_high - id: src7 title: "Dataloader 3.0: A new algorithm to solve the N+1 Problem" author: WunderGraph url: https://wundergraph.com/blog/dataloader_3_0_breadth_first_data_loading type: technical_blog published: 2024-11-05 reliability: moderate_high --- # Fix N+1 Queries in GraphQL with DataLoader ## TL;DR - **Bottom line**: Wrap every resolver that fetches related data in a DataLoader instance — it automatically batches individual `.load(id)` calls within one execution tick into a single bulk query, turning N+1 queries into 2. - **Key tool/command**: `new DataLoader(keys => batchFetchByIds(keys))` (JS) or `DataLoader(load_fn=batch_load)` (Python Strawberry) - **Watch out for**: Creating a single global DataLoader instance instead of one per request — this leaks cached data between users. - **Works with**: Any GraphQL server (Apollo Server 4, graphql-js, Strawberry, Graphene, gqlgen, graphql-java). DataLoader v2.x requires Node.js >= 10; Strawberry DataLoader requires Python >= 3.8 with asyncio. ## Constraints - DataLoader instances MUST be created per-request (in the GraphQL context factory) — a shared global instance caches data across users, causing data leaks and stale reads - The batch function MUST return results in the exact same order as the input keys array, with the same length — mismatches cause silent data corruption - DataLoader requires an async execution environment — Node.js event loop (JS) or asyncio event loop (Python) - Never disable batching (`batch: false`) in production — it defeats the entire purpose of DataLoader - Error objects returned in the batch array are cached by default — clear the cache on transient failures or disable error caching ## Quick Reference | # | Cause | Likelihood | Signature | Fix | |---|---|---|---|---| | 1 | No DataLoader — each resolver calls DB individually | ~50% of cases | N separate `SELECT ... WHERE id = ?` in query logs | Wrap in DataLoader with `WHERE id IN (?)` batch query | | 2 | DataLoader created globally, not per-request | ~15% of cases | Stale/wrong data returned; data leaks between users | Move DataLoader instantiation into context factory | | 3 | Batch function returns results in wrong order | ~10% of cases | Data assigned to wrong parent objects | Map results to input keys: `keys.map(k => resultMap.get(k))` | | 4 | Batch function returns wrong array length | ~8% of cases | `DataLoader must return array of same length` error | Return `null` for missing keys; match array length to keys | | 5 | Using `.loadMany()` where `.load()` suffices | ~5% of cases | Bypasses per-key deduplication within a tick | Use `.load()` in resolvers; `.loadMany()` only for explicit bulk | | 6 | Nested DataLoaders not batching across depth levels | ~5% of cases | Batch size = 1 for deeply nested fields | Ensure async resolvers; check `batchScheduleFn` timing | | 7 | Complex key objects not deduplicating | ~4% of cases | Duplicate DB queries despite same logical key | Provide `cacheKeyFn` that serializes keys to strings | | 8 | DataLoader cache hiding database updates | ~3% of cases | Mutations not reflected in subsequent queries | Call `loader.clear(key)` or `loader.clearAll()` after mutations | ## Decision Tree ``` START |-- Is your GraphQL server in JavaScript/TypeScript? | |-- YES --> Use `dataloader` npm package (src2). Go to "JS/TS: Apollo Server" code example. | +-- NO | |-- Is your server in Python with Strawberry? | |-- YES --> Use `strawberry.dataloader.DataLoader` (src4). Go to "Python: Strawberry" code example. | +-- NO | |-- Is your server in Python with Graphene? | |-- YES --> Use `aiodataloader` package (src5). Go to "Python: Graphene" code example. | +-- NO | |-- Is your server in Go (gqlgen)? | |-- YES --> Use `graph-gophers/dataloader` or gqlgen built-in dataloaden. | +-- NO | |-- Is your server using Apollo Federation? | |-- YES --> Apply DataLoader in each subgraph's reference resolver (src3). | +-- NO | +-- DEFAULT --> Implement the batch-and-cache pattern manually: collect keys during resolve phase, batch-fetch before returning. ``` ## Step-by-Step Guide ### 1. Identify the N+1 problem in your query logs Enable query logging in your database or GraphQL server to see repeated queries. Look for patterns where the same `SELECT` statement runs N times with different IDs. [src1] ```sql -- You'll see this pattern in logs (N+1): SELECT * FROM posts; -- 1 query SELECT * FROM users WHERE id = 1; -- N queries (one per post) SELECT * FROM users WHERE id = 2; SELECT * FROM users WHERE id = 3; -- ... repeated for every post.authorId ``` **Verify**: Count queries per GraphQL request in your DB logs. If `count > (unique tables queried * 2)`, you likely have N+1. ### 2. Install DataLoader Install the appropriate DataLoader package for your language. [src2] ```bash # JavaScript/TypeScript npm install dataloader # Python (Strawberry — built-in, no extra install) pip install strawberry-graphql # Python (Graphene) pip install aiodataloader ``` **Verify**: `npm list dataloader` → `dataloader@2.2.3` (or later) ### 3. Create a batch loading function The batch function receives an array of keys and must return a Promise/awaitable resolving to an array of results in the same order as the keys. [src2] ```javascript // JavaScript — batch loading function async function batchUsers(userIds) { // One query fetches ALL requested users const users = await db.query( 'SELECT * FROM users WHERE id = ANY($1)', [userIds] ); // CRITICAL: return results in same order as input keys const userMap = new Map(users.map(u => [u.id, u])); return userIds.map(id => userMap.get(id) || null); } ``` **Verify**: `batchUsers([3, 1, 2])` returns `[user3, user1, user2]` — same order as input. ### 4. Create DataLoader instances per-request in context Attach fresh DataLoader instances to the GraphQL context so every resolver in a single request shares the same loader (batching + dedup) but different requests get isolated caches. [src1] [src3] ```javascript // Apollo Server 4 — context factory const server = new ApolloServer({ typeDefs, resolvers }); const { url } = await startStandaloneServer(server, { context: async () => ({ loaders: { user: new DataLoader(batchUsers), post: new DataLoader(batchPosts), }, }), }); ``` **Verify**: Add `console.log('context created')` in the factory — it should log once per HTTP request. ### 5. Use DataLoader in resolvers Replace direct database calls in resolvers with `loader.load(key)`. [src1] ```javascript const resolvers = { Query: { posts: () => db.query('SELECT * FROM posts'), }, Post: { // BEFORE (N+1): db.query('SELECT * FROM users WHERE id = $1', [post.authorId]) // AFTER (batched): author: (post, _args, { loaders }) => loaders.user.load(post.authorId), }, }; ``` **Verify**: Run the query and check DB logs — you should see exactly 1 `SELECT * FROM posts` + 1 `SELECT * FROM users WHERE id = ANY(...)` instead of N+1 queries. ### 6. Clear cache after mutations After any create/update/delete operation, clear the relevant DataLoader cache to prevent stale reads within the same request. [src2] ```javascript const resolvers = { Mutation: { updateUser: async (_, { id, input }, { loaders }) => { const updated = await db.query( 'UPDATE users SET name = $1 WHERE id = $2 RETURNING *', [input.name, id] ); // Clear stale cache entry and prime with fresh data loaders.user.clear(id); loaders.user.prime(id, updated[0]); return updated[0]; }, }, }; ``` **Verify**: After mutation, subsequent resolvers in the same request return updated data. ## Code Examples ### JavaScript/TypeScript: Apollo Server 4 with DataLoader ```typescript // Input: GraphQL query { posts { id title author { id name } } } // Output: 2 DB queries total (1 for posts, 1 for all authors) instead of N+1 import { ApolloServer } from '@apollo/server'; import { startStandaloneServer } from '@apollo/server/standalone'; import DataLoader from 'dataloader'; // Batch function: keys in, values out (same order, same length) const batchUsers = async (ids: readonly number[]) => { const users = await db.query('SELECT * FROM users WHERE id = ANY($1)', [ids]); const map = new Map(users.rows.map((u: any) => [u.id, u])); return ids.map(id => map.get(id) || null); }; const resolvers = { Post: { author: (post: any, _: any, ctx: any) => ctx.loaders.user.load(post.author_id), }, }; const server = new ApolloServer({ typeDefs, resolvers }); const { url } = await startStandaloneServer(server, { context: async () => ({ loaders: { user: new DataLoader(batchUsers) }, }), }); ``` ### Python: Strawberry GraphQL with DataLoader > Full script: [python-strawberry-graphql-with-dataloader.py](scripts/python-strawberry-graphql-with-dataloader.py) (26 lines) ```python # Input: GraphQL query { posts { id title author { id name } } } # Output: 2 DB queries total instead of N+1 import strawberry from strawberry.dataloader import DataLoader from typing import List, Optional # ... (see full script) ``` ### Python: Graphene with aiodataloader ```python # Input: GraphQL query { posts { id title author { id name } } } # Output: 2 DB queries total instead of N+1 from aiodataloader import DataLoader import graphene class UserLoader(DataLoader): async def batch_load_fn(self, user_ids): users = {u.id: u for u in await User.objects.filter(id__in=user_ids)} return [users.get(uid) for uid in user_ids] # Preserve key order class PostType(graphene.ObjectType): id = graphene.Int() title = graphene.String() author = graphene.Field(lambda: UserType) async def resolve_author(self, info): return await info.context['user_loader'].load(self.author_id) # Create per-request in middleware/context def get_context(request): return {'user_loader': UserLoader()} ``` ## Anti-Patterns ### Wrong: Global singleton DataLoader (data leak between users) ```javascript // BAD — shared across all requests, leaks user A's data to user B const globalUserLoader = new DataLoader(batchUsers); const resolvers = { Post: { author: (post) => globalUserLoader.load(post.authorId), }, }; ``` ### Correct: Per-request DataLoader in context ```javascript // GOOD — fresh instance per request, isolated cache const server = new ApolloServer({ typeDefs, resolvers }); startStandaloneServer(server, { context: async () => ({ loaders: { user: new DataLoader(batchUsers) }, }), }); ``` ### Wrong: Batch function returns results in arbitrary order ```javascript // BAD — results don't match key order, causes data corruption async function batchUsers(ids) { const users = await db.query('SELECT * FROM users WHERE id = ANY($1)', [ids]); return users.rows; // Order depends on DB, NOT on input key order! } ``` ### Correct: Map results back to input key order ```javascript // GOOD — explicitly map to input key order async function batchUsers(ids) { const users = await db.query('SELECT * FROM users WHERE id = ANY($1)', [ids]); const map = new Map(users.rows.map(u => [u.id, u])); return ids.map(id => map.get(id) || null); // Same order as input } ``` ### Wrong: Calling the database directly inside every resolver ```javascript // BAD — each post triggers a separate DB query (N+1) const resolvers = { Post: { author: async (post) => { return await db.query('SELECT * FROM users WHERE id = $1', [post.authorId]); }, }, }; ``` ### Correct: Delegate to DataLoader ```javascript // GOOD — DataLoader batches all author lookups into one query const resolvers = { Post: { author: (post, _, { loaders }) => loaders.user.load(post.authorId), }, }; ``` ### Wrong: Using loadMany in individual resolvers instead of load ```javascript // BAD — loadMany in a loop bypasses automatic batching const resolvers = { Query: { users: async (_, { ids }, { loaders }) => { // This fetches all at once but skips per-key dedup in nested resolvers return await loaders.user.loadMany(ids); }, }, Post: { // Using loadMany for a single key is wasteful author: (post, _, { loaders }) => loaders.user.loadMany([post.authorId]).then(r => r[0]), }, }; ``` ### Correct: Use load() for single keys, loadMany() for explicit bulk ```javascript // GOOD — load() for resolvers, loadMany() only for explicit batch operations const resolvers = { Query: { users: (_, { ids }, { loaders }) => loaders.user.loadMany(ids), // Correct: explicit bulk }, Post: { author: (post, _, { loaders }) => loaders.user.load(post.authorId), // Correct: single key }, }; ``` ## Common Pitfalls - **Batch function returns wrong length**: DataLoader throws `DataLoader must be constructed with a function which accepts Array and returns Promise>, but the function did not return a Promise of an Array of the same length as the Array of keys`. Fix: ensure you return `null` for missing keys — `keys.map(k => map.get(k) || null)`. [src2] - **Cache persists across requests**: Users see other users' data because DataLoader instance lives beyond a single request. Fix: always create DataLoader in the context factory function, never at module scope. [src1] - **Errors are cached by default**: A transient DB error gets cached, and all subsequent `.load()` calls for that key return the cached error. Fix: call `loader.clear(key)` in your error handler, or use `new DataLoader(fn, { cache: false })` for volatile data. [src2] - **Object keys don't deduplicate**: Two `{id: 1}` objects are different references, so DataLoader treats them as different keys. Fix: provide `cacheKeyFn: (key) => key.id` or serialize to string. [src2] - **Async/await breaks batching window**: If you `await` before calling `.load()`, the batching tick may have already fired. Fix: collect all `.load()` calls synchronously within a resolver, or use `Promise.all()` for parallel loads. [src1] - **Federation subgraphs missing DataLoader**: Each subgraph's `__resolveReference` runs once per entity key, recreating the N+1 problem at the subgraph boundary. Fix: use DataLoader in `__resolveReference` resolvers too. [src3] - **No DataLoader for mutations**: After a mutation, subsequent reads in the same request return stale cached data. Fix: call `loader.clear(key)` after writes, optionally `loader.prime(key, newValue)`. [src2] - **maxBatchSize not set for large lists**: Batch functions may receive thousands of keys, causing `IN (...)` clauses that exceed DB limits. Fix: set `maxBatchSize: 100` (or your DB's practical limit). [src2] ## Diagnostic Commands ```bash # Check if DataLoader is installed (Node.js) npm list dataloader # Expected: dataloader@2.2.3 # Check for N+1 in PostgreSQL logs (enable statement logging first) # In postgresql.conf: log_statement = 'all' grep 'SELECT.*FROM users WHERE id =' /var/log/postgresql/postgresql.log | wc -l # If count >> number of unique IDs, you have N+1 # Enable Apollo Server query logging to detect N+1 # Set DEBUG=knex:query (if using Knex) or enable morgan logging DEBUG=knex:query node server.js # Profile GraphQL execution time per resolver (Apollo Studio) # Or use Apollo Server plugin: ApolloServerPluginUsageReporting # Check DataLoader batch sizes at runtime (add to batch function) # async function batchUsers(ids) { # console.log(`DataLoader batch size: ${ids.length}`); # ... # } # Python: check aiodataloader version pip show aiodataloader # Expected: aiodataloader >= 0.2.0 ``` ## Version History & Compatibility | Version | Status | Breaking Changes | Migration Notes | |---|---|---|---| | dataloader 2.2.x (JS) | Current (2024) | None | Stable since 2.0.0 | | dataloader 2.0.0 (JS) | LTS | Dropped Node < 10; TypeScript rewrite | Update Node.js; types now built-in | | dataloader 1.x (JS) | EOL | — | Upgrade: API is the same, just bump version | | strawberry.dataloader (Python) | Current (2025) | None | Built into strawberry-graphql >= 0.100.0 | | aiodataloader 0.2.x (Python) | Current (2024) | None | For Graphene; requires asyncio | | graphql-batch (Ruby) | Current (2024) | None | Shopify's alternative; different API | ## When to Use / When Not to Use | Use When | Don't Use When | Use Instead | |---|---|---| | Nested GraphQL resolvers fetch related entities by ID | Single resolver fetches one record | Direct database query | | Same entity is requested multiple times in one query | Data changes between resolver calls within one request | Disable cache or clear after mutation | | Federated subgraph resolves entity references | You need cross-request persistent caching | Redis/Memcached layer | | Any list-to-detail resolver pattern (posts -> authors) | Query complexity is bounded and small (< 5 items) | Direct fetch is fine for small N | | You want to decouple data fetching from schema structure | ORM already handles eager loading (e.g., Rails includes) | Use ORM's built-in batching | ## Important Caveats - DataLoader only caches within a single request — it is NOT a replacement for Redis/Memcached. For cross-request caching, layer a persistent cache underneath your batch function. - The batch function's returned array MUST match the length and order of the input keys array. This is the most common source of bugs. Always build a map from results and reconstruct the array from keys. - In serverless environments (AWS Lambda, Cloudflare Workers), DataLoader instances are naturally per-invocation, but be careful with warm starts that might reuse module-level variables. - Disabling the cache (`cache: false`) means duplicate keys within the same tick will appear multiple times in the batch function's keys array — your batch function must handle duplicates. - DataLoader uses `process.nextTick` (Node.js) or microtask scheduling by default. Custom `batchScheduleFn` can change this but may break batching if the window is too short. ## Related Units - [Database Connection Pooling](/software/devops/database-connection-pooling/2026) - [Caching Strategies for APIs](/software/patterns/api-caching-strategies/2026)