ERP Bulk Import Comparison: Bulk API 2.0 vs IDoc vs FBDI vs SuiteTalk vs DMF

TL;DR

• Bottom line: Salesforce Bulk API 2.0 is the most mature and developer-friendly bulk interface; SAP IDoc handles highest volume but is sunset in cloud editions; Oracle FBDI is template-driven with module-specific limits; NetSuite SuiteTalk is the most constrained at 1,000 records/request; Dynamics 365 DMF offers the most configurability for tuning parallel imports.
• Key limit: Salesforce caps at 150 MB/job and 150M records/24h; SAP IDoc recommends <10 MB per document; NetSuite SOAP maxes at 1,000 records/call with 5-20 concurrent connections; D365 DMF defaults to 8 parallel threads but is configurable.
• Watch out for: SAP IDocs are no longer supported in SAP Cloud ERP (public cloud) starting August 2025 release 2508 — only OData/SOAP/REST APIs remain. [src3]
• Best for: High-volume data migrations (>100K records), scheduled batch imports, initial system loads, and recurring bulk synchronizations between ERPs and external systems.
• Authentication: Each system uses its own auth — Salesforce OAuth 2.0, SAP certificate/SSO, Oracle OAuth 2.0, NetSuite TBA/OAuth 2.0, D365 Microsoft Entra ID.

System Profile

This comparison covers the five most widely adopted ERP bulk import mechanisms. Each system takes a fundamentally different architectural approach: Salesforce uses an asynchronous REST-based job model, SAP uses a document-oriented messaging protocol (IDoc/ALE), Oracle Fusion uses file-based templates processed via ESS jobs, NetSuite uses SOAP web services with governance-controlled concurrency, and Microsoft uses a configurable framework with parallel staging-to-target processing.

API Surfaces & Capabilities

Rate Limits & Quotas

Per-Request Limits

Rolling / Daily Limits

Concurrency Limits

Authentication

Authentication Gotchas

• SAP IDoc partner profile misconfiguration is the #1 cause of "IDoc stuck in status 30" errors — verify RFC destination connection test first [src3]
• NetSuite TBA tokens expire if the role is inactivated or the employee record is modified — use a dedicated integration role [src5]
• D365 user-based throttling tracks by user object ID + app client ID combination — one service account can still hit limits across multiple integrations [src6]

Constraints

• SAP IDocs are NOT available in SAP S/4HANA Public Cloud (Cloud ERP) starting release 2508 (August 2025) — only OData, SOAP, and REST APIs are supported [src3]
• Salesforce Bulk API 2.0 does not support real-time operations — minimum latency is seconds to minutes depending on queue depth [src1]
• Oracle FBDI requires module-specific Excel templates — you cannot use arbitrary CSV formats [src4]
• NetSuite REST API can only create/update one record per request — use SuiteTalk SOAP for bulk (1,000/call) or CSV import (25,000/file) [src5]
• D365 DMF is exempt from service protection API limits, but OData-based integrations are NOT exempt [src6]
• NetSuite concurrency is shared across ALL integration types (SOAP, REST, RESTlet) — a single high-volume integration can starve others [src5]
• D365 DMF requires batch mode for parallel processing — running without batch uses a single thread [src7]

Integration Pattern Decision Tree

START -- User needs to bulk import data into an ERP
|
+-- Which ERP system?
|   +-- Salesforce
|   |   +-- Records > 2,000? --> Bulk API 2.0 (CSV upload, async)
|   |   +-- Records < 2,000? --> REST API (composite, sync)
|   |   +-- Migration > 10M records? --> Bulk API 2.0 with job chunking (150 MB/job)
|   +-- SAP S/4HANA
|   |   +-- On-premise / Private Cloud?
|   |   |   +-- Structured EDI/master data --> IDoc/ALE (packet size 50-100)
|   |   |   +-- Custom data --> BAPI via RFC
|   |   |   +-- Modern API --> OData with $batch
|   |   +-- Public Cloud (Cloud ERP)?
|   |       +-- IDocs NOT available --> OData or SOAP APIs only
|   +-- Oracle ERP Cloud (Fusion)
|   |   +-- Standard import process exists? --> FBDI
|   |   +-- Custom data entity? --> REST API
|   |   +-- HCM data? --> HCM Data Loader (HDL)
|   +-- NetSuite
|   |   +-- Volume < 25K records? --> CSV Import or SuiteTalk SOAP (1K/call)
|   |   +-- Volume > 25K records? --> Split into 25K chunks
|   +-- Dynamics 365 F&O
|       +-- Data migration? --> DMF with batch mode, 8-16 threads
|       +-- Recurring? --> Recurring integrations (exempt from API limits)
|       +-- Ad-hoc API? --> OData $batch (max 5,000 ops)
+-- Cross-system bulk transfer?
    +-- Source ERP --> flat file/CSV --> Target ERP bulk import
    +-- Or use middleware (MuleSoft, Boomi, Workato)

Quick Reference

Error Handling & Failure Points

Common Error Codes

Failure Points in Production

• Salesforce CSV encoding: Bulk API 2.0 jobs produce incorrect data with BOM characters or non-UTF-8. Fix: Always strip BOM and enforce UTF-8 encoding before upload. [src1]
• SAP IDoc packet size = 1: Default processes one IDoc per transaction; 100K+ IDocs exhaust dialog processes. Fix: Increase packet size to 50-100 in partner profile (WE20) and enable background processing. [src3]
• Oracle FBDI wrong template version: Using a previous release template causes silent column misalignment. Fix: Re-download templates from target environment before each import cycle. [src4]
• NetSuite concurrency starvation: A single integration consuming all slots blocks everything else. Fix: Allocate max 10 of 15 slots to any single integration; use governanceLimits endpoint to monitor. [src5]
• D365 DMF single-thread execution: Running without batch mode uses only one thread. Fix: Always execute in batch mode with dedicated batch group. [src7]
• D365 Excel driver contention: Excel files can fail after 10-minute timeout. Fix: Convert Excel to CSV before import. [src7]

Anti-Patterns

Wrong: Using NetSuite REST API for bulk operations

// BAD -- REST API processes one record at a time
for (const record of records) {
  await fetch(`https://${accountId}.suitetalk.api.netsuite.com/...`, {
    method: 'POST', body: JSON.stringify(record)
  });
}
// 10,000 records = 10,000 API calls = hits concurrency + rate limits

Correct: Use SuiteTalk SOAP addList for bulk operations

// GOOD -- SOAP addList processes up to 1,000 records per call
const batchSize = 1000;
for (let i = 0; i < records.length; i += batchSize) {
  const batch = records.slice(i, i + batchSize);
  await soapClient.addList({ record: batch });
}
// 10,000 records = 10 API calls

Wrong: Running D365 DMF imports without batch mode

// BAD -- Direct execution uses single thread
// Data Management > Import > Execute (without batch)
// 100K records processed sequentially = hours

Correct: Execute DMF in batch mode with parallel threads

// GOOD -- Enable batch mode with dedicated batch group
// Import threshold record count: 2,500
// Import task count: 4-8
// 100K records / 4 threads = 4x faster

Wrong: SAP IDoc with default packet size for high-volume loads

// BAD -- Default packet size = 1
// 50,000 IDocs = 50,000 transactions = dialog process exhaustion

Correct: Configure packet processing with appropriate batch size

// GOOD -- Set packet size 50-100 in WE20 partner profile
// 50,000 IDocs / 100 = 500 transactions
// Enable background processing: BD87 batch job

Common Pitfalls

• Salesforce: Not checking per-record results: Bulk API 2.0 jobs can return jobComplete with partial failures. Fix: Always poll both successfulResults and failedResults endpoints. [src1]
• SAP: Assuming IDoc availability in cloud: Cloud ERP (2508+) dropped IDoc support. Fix: Audit IDoc interfaces; migrate to OData $batch or SOAP before cloud migration. [src3]
• Oracle FBDI: Skipping validation step: Errors only visible after ESS job execution. Fix: Pre-validate data against published field requirements before upload. [src4]
• NetSuite: Ignoring governance unit costs: Different operations consume different units. Fix: Calculate total governance cost before bulk submission; prefer SuiteQL for reads. [src5]
• D365: Not disabling change tracking during migration: Adds overhead to every insert/update. Fix: Disable change tracking before migration; re-enable after. [src7]
• Cross-system: Assuming same date/time formats: Salesforce UTC, SAP local timezone, NetSuite user preference. Fix: Normalize all datetime values to UTC/ISO 8601 in integration layer.

Diagnostic Commands

# === Salesforce Bulk API 2.0 ===
# Check job status
curl -H "Authorization: Bearer $SF_TOKEN" \
  "https://yourinstance.salesforce.com/services/data/v62.0/jobs/ingest/$JOB_ID"

# Get failed records
curl -H "Authorization: Bearer $SF_TOKEN" \
  "https://yourinstance.salesforce.com/services/data/v62.0/jobs/ingest/$JOB_ID/failedResults"

# Check API usage limits
curl -H "Authorization: Bearer $SF_TOKEN" \
  "https://yourinstance.salesforce.com/services/data/v62.0/limits"

# === SAP IDoc ===
# Check IDoc status: transaction WE02 or WE05
# Reprocess failed IDocs: transaction BD87
# Test RFC destination: transaction SM59

# === Oracle FBDI ===
# Check ESS job status
curl -H "Authorization: Bearer $OCI_TOKEN" \
  "https://instance.fa.ocs.oraclecloud.com/hcmRestApi/resources/.../erpintegrations?finder=ESSJobStatusFinder;requestId=$JOB_ID"

# === NetSuite ===
# Check governance limits
curl -H "Authorization: OAuth ..." \
  "https://$ACCOUNT.suitetalk.api.netsuite.com/services/rest/record/v1/metadata-catalog/governanceLimits"

# === D365 F&O ===
# Monitor via LCS or OData
curl -H "Authorization: Bearer $D365_TOKEN" \
  "https://instance.operations.dynamics.com/data/DataManagementDefinitionGroups"

Version History & Compatibility

When to Use / When Not to Use

Cross-System Comparison

Important Caveats

• Rate limits and concurrency quotas vary significantly by edition/tier — always verify limits against your specific license agreement
• SAP IDoc sunset in public cloud is the most significant breaking change — organizations planning SAP cloud migration must budget for interface redesign
• Oracle FBDI templates change with each quarterly release — never cache templates across environment updates
• NetSuite's shared concurrency pool means bulk operations directly compete with real-time UI and REST API usage by end users
• D365 service protection limits are resource-based and can vary between environments — Microsoft explicitly states these limits "can change and might vary among environments" [src6]
• All performance numbers represent documented limits, not guaranteed throughput — actual performance depends on data complexity, validations, customizations, and system load