ERP API Pagination Patterns Comparison: Cursor vs Offset vs Keyset vs nextRecordsUrl

TL;DR

• Bottom line: Every major ERP uses a different pagination pattern -- Salesforce uses server-side cursor (nextRecordsUrl), SAP and D365 use OData $skiptoken, Oracle uses offset/limit, and NetSuite uses offset with a hard 1,000-page cap. Cursor-based pagination scales; offset-based breaks past 100K records.
• Key limit: NetSuite caps total retrievable records at pageSize x 1,000 pages -- if you have 500K records with page size 100, you can only retrieve 100K.
• Watch out for: Offset pagination under concurrent writes causes duplicate or missing records on every ERP -- Salesforce and D365 solve this with cursor/skiptoken; SAP, Oracle, NetSuite do not by default.
• Best for: This card answers "which pagination approach does my target ERP support and how does it scale?" -- use before designing any data extraction integration.
• Authentication: Pagination tokens are session-scoped -- if your OAuth token expires mid-pagination, most ERPs invalidate the cursor/offset state and you must restart.

System Profile

This comparison card covers pagination patterns across 8 ERP systems. The focus is on outbound data retrieval pagination (reading data from the ERP), not inbound bulk import patterns.

API Surfaces & Capabilities

Rate Limits & Quotas

Pagination-Specific Limits

Concurrent Pagination Sessions

Authentication

Authentication Gotchas

• Salesforce query locators expire after 15 minutes of inactivity regardless of OAuth token validity. [src1]
• Dynamics 365 @odata.nextLink URLs embed server-specific routing -- do NOT modify the nextLink URL. [src5]
• NetSuite TBA signatures must be regenerated per request -- cannot reuse Authorization header across pages. [src4]

Constraints

• Salesforce: Page size is server-controlled at 2,000 records -- cannot request larger pages
• SAP S/4HANA: Client-driven $skip/$top is not safe under concurrent writes -- rows shift
• Oracle ERP Cloud: Results using offset/limit are NOT ordered by default -- must add orderBy
• NetSuite: 1,000-page cap means max records = page_size x 1,000
• Dynamics 365: $skip is deprecated for server-driven paging in favor of $skiptoken
• Workday: RaaS has zero pagination -- reports >50K rows timeout
• Acumatica: No hard limit on $skip but database scan time grows linearly
• All systems: Offset-based pagination has O(n) performance degradation

Integration Pattern Decision Tree

START -- Need to paginate through ERP data
|
+-- Which ERP?
|   +-- Salesforce --> nextRecordsUrl cursor (2,000/page, O(1))
|   |   +-- >10M records? --> Bulk API 2.0 instead
|   +-- SAP S/4HANA --> Server-driven $skiptoken (Prefer header)
|   |   +-- >100K records? --> CDS views or BICC
|   +-- Oracle ERP Cloud --> offset/limit + orderBy (REQUIRED)
|   |   +-- >100K records? --> BICC or BI Publisher
|   +-- NetSuite --> offset/limit (1,000 page cap!)
|   |   +-- >pageSize*1000? --> Date-range chunking
|   +-- Dynamics 365 --> Follow @odata.nextLink (never construct manually)
|   |   +-- >500K records? --> Data Management Framework
|   +-- Workday REST --> page/offset (max 100/page)
|   |   +-- RaaS reports --> No pagination; chunk with prompts
|   +-- IFS Cloud --> OData V4 server-driven paging
|   +-- Acumatica --> <50K: $top/$skip; >50K: filter-based keyset
|
+-- Performance: <10K: any method; 10K-100K: prefer cursor; >100K: cursor required; >1M: bulk API

Quick Reference

Cross-System Pagination Comparison

Pagination Pattern Types

Step-by-Step Integration Guide

1. Salesforce: Paginate with nextRecordsUrl

Salesforce REST API returns a nextRecordsUrl field when results exceed 2,000 records. This URL contains a server-side query locator acting as a cursor. [src1]

import requests

def paginate_salesforce(instance_url, access_token, soql_query):
    headers = {"Authorization": f"Bearer {access_token}"}
    url = f"{instance_url}/services/data/v62.0/query"
    params = {"q": soql_query}
    all_records = []
    while True:
        resp = requests.get(url, headers=headers, params=params)
        resp.raise_for_status()
        data = resp.json()
        all_records.extend(data["records"])
        if data["done"]:
            break
        url = f"{instance_url}{data['nextRecordsUrl']}"
        params = {}
    return all_records

Verify: data["totalSize"] should match len(all_records)

2. SAP S/4HANA: Server-Driven $skiptoken Paging

Set Prefer: odata.maxpagesize=N header. Response includes @odata.nextLink with $skiptoken. [src2]

import requests

def paginate_sap_odata(base_url, access_token, entity_set, page_size=1000):
    headers = {
        "Authorization": f"Bearer {access_token}",
        "Prefer": f"odata.maxpagesize={page_size}",
        "Accept": "application/json"
    }
    url = f"{base_url}/sap/opu/odata4/sap/{entity_set}"
    all_records = []
    while url:
        resp = requests.get(url, headers=headers)
        resp.raise_for_status()
        data = resp.json()
        all_records.extend(data.get("value", []))
        url = data.get("@odata.nextLink")  # Don't modify this URL
    return all_records

Verify: Compare count against GET {entity_set}/$count

3. Oracle ERP Cloud: Offset/Limit with orderBy

Always include orderBy -- results are non-deterministic without it. [src3]

import requests

def paginate_oracle_erp(base_url, access_token, resource, page_size=500):
    headers = {"Authorization": f"Bearer {access_token}"}
    all_records, offset = [], 0
    while True:
        params = {"limit": page_size, "offset": offset,
                  "orderBy": "CreationDate:asc", "totalResults": "true"}
        resp = requests.get(f"{base_url}/fscmRestApi/resources/latest/{resource}",
                           headers=headers, params=params)
        resp.raise_for_status()
        data = resp.json()
        all_records.extend(data.get("items", []))
        if not data.get("hasMore", False):
            break
        offset += page_size
    return all_records

Verify: data["totalResults"] matches len(all_records)

4. Dynamics 365: Follow @odata.nextLink

D365 returns @odata.nextLink with $skiptoken. Never construct pagination URLs manually. [src5, src8]

import requests

def paginate_dynamics365(base_url, access_token, entity, max_page_size=10000):
    headers = {"Authorization": f"Bearer {access_token}",
               "Prefer": f"odata.maxpagesize={max_page_size}"}
    url = f"{base_url}/data/{entity}"
    all_records = []
    while url:
        resp = requests.get(url, headers=headers)
        resp.raise_for_status()
        data = resp.json()
        all_records.extend(data.get("value", []))
        url = data.get("@odata.nextLink")
    return all_records

Verify: GET {entity}/$count matches len(all_records)

5. Acumatica: Filter-Based Keyset for Large Datasets

For >50K records, avoid $skip and use filter-based keyset by sorting on a unique field.

import requests

def paginate_acumatica_keyset(base_url, cookies, entity, page_size=100):
    all_records, last_id = [], ""
    while True:
        params = {"$top": page_size, "$orderby": "InventoryID"}
        if last_id:
            params["$filter"] = f"InventoryID gt '{last_id}'"
        resp = requests.get(f"{base_url}/entity/Default/24.200.001/{entity}",
                           cookies=cookies, params=params)
        resp.raise_for_status()
        records = resp.json()
        if not records:
            break
        all_records.extend(records)
        last_id = records[-1]["InventoryID"]["value"]
    return all_records

Verify: Compare against Acumatica Generic Inquiry count

Code Examples

cURL: Test Pagination for Each ERP

# Salesforce: Query with cursor pagination
curl -s -H "Authorization: Bearer $SF_TOKEN" \
  "$SF_INSTANCE/services/data/v62.0/query?q=SELECT+Id,Name+FROM+Account" \
  | jq '{totalSize: .totalSize, done: .done, nextUrl: .nextRecordsUrl}'

# SAP S/4HANA: Server-driven paging
curl -s -H "Authorization: Bearer $SAP_TOKEN" \
  -H "Prefer: odata.maxpagesize=100" \
  "$SAP_URL/sap/opu/odata4/sap/API_BUSINESS_PARTNER/A_BusinessPartner" \
  | jq '{count: (.value | length), nextLink: ."@odata.nextLink"}'

# Oracle ERP Cloud: Offset pagination (always include orderBy!)
curl -s -H "Authorization: Bearer $ORA_TOKEN" \
  "$ORA_URL/fscmRestApi/resources/latest/invoices?limit=500&offset=0&orderBy=InvoiceId:asc&totalResults=true" \
  | jq '{totalResults: .totalResults, hasMore: .hasMore}'

# NetSuite: Check for 1,000-page cap risk
curl -s -H "Authorization: $NS_AUTH" \
  "https://$NS_ACCOUNT.suitetalk.api.netsuite.com/services/rest/record/v1/customer?limit=1" \
  | jq '.totalResults'

# Dynamics 365: Follow @odata.nextLink
curl -s -H "Authorization: Bearer $D365_TOKEN" \
  -H "Prefer: odata.maxpagesize=5000" \
  "$D365_URL/data/SalesOrderHeaders" \
  | jq '{count: (.value | length), nextLink: ."@odata.nextLink"}'

Error Handling & Failure Points

Common Error Codes

Failure Points in Production

• Salesforce cursor expiry mid-extraction: Downstream processing taking >15 min per page kills the cursor. Fix: Buffer all pages locally first, then process. [src1]
• SAP duplicate records with $skip/$top: Concurrent writes shift row positions. Fix: Use Prefer: odata.maxpagesize=N for $skiptoken-based paging. [src2]
• Oracle silent data inconsistency: No orderBy causes non-deterministic results. Fix: Always orderBy on unique, immutable column. [src3]
• NetSuite 1,000-page ceiling: Integration silently stops at page 1,000. Fix: Date-range chunking within lastmodifieddate windows. [src4]
• D365 $skip deprecation breakage: Manual $skip construction breaks on update. Fix: Always follow @odata.nextLink from response. [src5, src8]
• Workday RaaS timeout: Reports >50K rows fail before returning data. Fix: Add prompts (date ranges) to chunk, or use WQL API. [src6]

Anti-Patterns

Wrong: Using $skip/$top for deep pagination on SAP

# BAD -- O(n) scan + data consistency issues at high offsets
offset = 0
while True:
    url = f"{base_url}/API_BUSINESS_PARTNER?$top=1000&$skip={offset}"
    data = requests.get(url, headers=headers).json()
    records.extend(data["value"])
    if len(data["value"]) < 1000: break
    offset += 1000  # At 500K: 30+ second queries

Correct: Server-driven $skiptoken paging

# GOOD -- O(1) per page, consistent results
headers["Prefer"] = "odata.maxpagesize=1000"
url = f"{base_url}/API_BUSINESS_PARTNER"
while url:
    data = requests.get(url, headers=headers).json()
    records.extend(data["value"])
    url = data.get("@odata.nextLink")  # Contains $skiptoken

Wrong: Oracle offset without orderBy

# BAD -- non-deterministic results; records may duplicate or disappear
params = {"limit": 500, "offset": offset}
data = requests.get(f"{url}/invoices", params=params).json()

Correct: Always include orderBy

# GOOD -- deterministic page ordering on unique column
params = {"limit": 500, "offset": offset, "orderBy": "InvoiceId:asc"}
data = requests.get(f"{url}/invoices", params=params).json()

Wrong: Ignoring NetSuite 1,000-page cap

# BAD -- silently stops at page 1,000 even with more data
while True:
    data = requests.get(f"{url}?limit=100&offset={offset}").json()
    all_records.extend(data["items"])
    if not data["hasMore"]: break  # False positive at page 1,000!
    offset += 100

Correct: Date-range chunking for NetSuite

# GOOD -- chunk by date to stay under 1,000-page cap per window
current = start_date
while current < end_date:
    chunk_end = min(current + timedelta(days=7), end_date)
    records = paginate_within_cap(
        f"{url}?q=lastModifiedDate BETWEEN '{current}' AND '{chunk_end}'",
        page_size=1000)
    all_records.extend(records)
    current = chunk_end

Common Pitfalls

• Assuming all ERPs support cursor pagination: Only Salesforce and OData-based systems (SAP, D365, IFS) offer true cursor/$skiptoken. Oracle and NetSuite are offset-only. Fix: Check this comparison table before designing integration. [src7]
• Constructing pagination URLs manually: D365 and SAP nextLink URLs are opaque. Fix: Use the complete nextLink URL as-is. [src5, src8]
• Not handling token expiry during long pagination: 500K records on Salesforce takes ~250 pages. Fix: Refresh tokens proactively before expiry. [src1]
• Using max page sizes to "optimize": 10,000/page on D365 increases timeout risk. Fix: Use 500-2,000 for reliable throughput. [src5]
• Ignoring the NetSuite 1,000-page cap: hasMore=false is a false negative at page 1,000. Fix: Verify retrieved count; implement date-range chunking. [src4]
• Running offset pagination on volatile data: Inserts/deletes between requests cause gaps. Fix: Use cursor where available; paginate on immutable columns otherwise. [src7]

Diagnostic Commands

# Salesforce: Pre-check record count
curl -s -H "Authorization: Bearer $SF_TOKEN" \
  "$SF_INSTANCE/services/data/v62.0/query?q=SELECT+COUNT()+FROM+Account" \
  | jq '.records[0].expr0'

# SAP S/4HANA: Get entity count
curl -s -H "Authorization: Bearer $SAP_TOKEN" \
  "$SAP_URL/sap/opu/odata4/sap/API_BUSINESS_PARTNER/A_BusinessPartner/\$count"

# Oracle ERP Cloud: Get total results
curl -s -H "Authorization: Bearer $ORA_TOKEN" \
  "$ORA_URL/fscmRestApi/resources/latest/invoices?limit=1&totalResults=true" \
  | jq '.totalResults'

# NetSuite: Assess page cap risk
curl -s -H "Authorization: $NS_AUTH" \
  "https://$NS_ACCOUNT.suitetalk.api.netsuite.com/services/rest/record/v1/customer?limit=1" \
  | jq '.totalResults'

# Dynamics 365: Count entities
curl -s -H "Authorization: Bearer $D365_TOKEN" \
  "$D365_URL/data/SalesOrderHeaders/\$count"

Version History & Compatibility

When to Use / When Not to Use

Cross-System Comparison

*Subject to daily API call quota. **Subject to timeout at very high offsets.

Important Caveats

• Pagination patterns change with API versions -- SAP's shift from $skip to $skiptoken and D365's deprecation of manual $skip are recent examples.
• Performance numbers (O(1) vs O(n)) are theoretical -- actual performance depends on indexes, query complexity, and server load.
• "Stateless" offset pagination is only safe if data does not change between requests.
• NetSuite's 1,000-page cap is not documented prominently -- many developers discover it in production.
• Workday RaaS "no pagination" is architectural, not a bug -- use WQL or smaller scopes.
• Token/cursor expiry times may be configurable by admins -- never hardcode assumptions.
• This covers cloud/SaaS deployments. On-premise versions may differ.