---
# === IDENTITY ===
id: business/erp-integration/change-data-capture-erp/2026
canonical_question: "How does Change Data Capture work for ERP integration - Debezium, GoldenGate, cloud ERP limitations?"
aliases:
  - "CDC for ERP integration patterns and tools"
  - "Debezium vs GoldenGate for ERP data replication"
  - "Change Data Capture limitations in cloud ERP systems"
  - "Log-based CDC vs polling for ERP synchronization"
entity_type: erp_integration
domain: business > erp-integration > change-data-capture-erp
region: global
jurisdiction: global
temporal_scope: 2024-2026

# === SYSTEM PROFILE ===
systems:
  - name: "Debezium"
    vendor: "Red Hat / Community"
    version: "3.4.x (Dec 2025)"
    edition: "Open Source (Apache 2.0)"
    deployment: "cloud | on-premise | hybrid"
    api_surface: "Kafka Connect / Debezium Server / Embedded Engine"
  - name: "Oracle GoldenGate"
    vendor: "Oracle"
    version: "23ai / 21c"
    edition: "Enterprise (licensed per processor)"
    deployment: "cloud | on-premise | hybrid"
    api_surface: "Trail files / REST API / Oracle Cloud Infrastructure"
  - name: "Salesforce Change Data Capture"
    vendor: "Salesforce"
    version: "API v66.0 (Spring '26)"
    edition: "Enterprise+ (included) / Developer"
    deployment: "cloud"
    api_surface: "Pub/Sub API / CometD Streaming"
  - name: "SAP ODP / SLT"
    vendor: "SAP"
    version: "S/4HANA 2023+"
    edition: "All editions with ODP support"
    deployment: "cloud | on-premise | hybrid"
    api_surface: "ODP Framework / RFC / CDS Views"

# === VERIFICATION ===
last_verified: 2026-03-02
confidence: 0.88
version: 1.0
first_published: 2026-03-02

# === TEMPORAL VALIDITY ===
temporal_validity:
  status: evolving
  last_breaking_change: "Debezium 3.0 (Oct 2024) — Java 17+ required, Kafka 3.x baseline"
  next_review: 2026-08-29
  change_sensitivity: high

# === CONSTRAINTS ===
constraints:
  - "Cloud SaaS ERPs (Salesforce, Workday, NetSuite) do not expose database transaction logs — log-based CDC is impossible without vendor-native event streams"
  - "Oracle GoldenGate requires $17,500/processor list price on BOTH source and target — licensing alone can exceed $100K for mid-size deployments"
  - "Salesforce CDC retains events for 3 days only — subscribers that fall behind lose data permanently"
  - "Debezium requires Kafka infrastructure (or Debezium Server) — minimum viable cluster is 3 brokers + ZooKeeper/KRaft"
  - "SAP ODP delta extraction requires SAP LT Replication Server (SLT) as proxy for application table CDC — adds infrastructure and licensing"
  - "Log-based CDC on Oracle requires supplemental logging enabled — adds 5-15% write overhead to redo logs"

# === SKIP CONDITIONS ===
skip_this_unit_if:
  - condition: "Need real-time CDC specifically within Salesforce ecosystem only"
    use_instead: "business/erp-integration/salesforce-platform-events/2026"
  - condition: "Need SAP-to-Azure specific CDC pipeline"
    use_instead: "business/erp-integration/sap-azure-data-factory-cdc/2026"
  - condition: "Need simple polling-based ERP sync for <1,000 records/day"
    use_instead: "business/erp-integration/erp-api-polling-patterns/2026"

# === AGENT HINTS ===
inputs_needed:
  - key: source_erp
    question: "Which ERP system is the CDC source?"
    type: choice
    options:
      - "Oracle EBS / E-Business Suite (on-premise)"
      - "SAP S/4HANA or ECC (on-premise or private cloud)"
      - "Salesforce (cloud)"
      - "NetSuite (cloud)"
      - "Workday (cloud)"
      - "Microsoft Dynamics 365 (cloud/hybrid)"
      - "Other on-premise ERP with relational DB"
  - key: cdc_method
    question: "Do you have direct database access for log-based CDC?"
    type: choice
    options:
      - "Yes — full DB admin access (on-premise / IaaS)"
      - "Partial — managed database with log access (RDS, Cloud SQL)"
      - "No — SaaS ERP, no DB access"
  - key: data_volume
    question: "What's your daily change volume?"
    type: choice
    options:
      - "< 10,000 changes/day"
      - "10,000-1,000,000 changes/day"
      - "> 1,000,000 changes/day"
  - key: latency_requirement
    question: "What's your acceptable replication latency?"
    type: choice
    options:
      - "Sub-second (real-time)"
      - "Seconds to minutes (near real-time)"
      - "Minutes to hours (batch-acceptable)"

# === DISTRIBUTION ===
canonical_source: "https://knowledgelib.io/business/erp-integration/change-data-capture-erp/2026"
suggested_citation: "Source: knowledgelib.io — AI Knowledge Library (verified 2026-03-02)"

# === RELATED UNITS ===
related_kos:
  depends_on: []
  related_to:
    - id: "business/erp-integration/salesforce-platform-events/2026"
      label: "Salesforce Platform Events and CDC"
    - id: "business/erp-integration/sap-azure-data-factory-cdc/2026"
      label: "SAP CDC with Azure Data Factory"
  solves:
    - id: "business/erp-integration/erp-real-time-sync-patterns/2026"
      label: "Real-time ERP synchronization patterns"
  alternative_to:
    - id: "business/erp-integration/erp-api-polling-patterns/2026"
      label: "API polling for ERP integration (simpler but less efficient)"
  often_confused_with:
    - id: "business/erp-integration/etl-batch-erp-integration/2026"
      label: "ETL batch extraction (scheduled, not event-driven)"

# === SOURCES ===
sources:
  - id: src1
    title: "Debezium Architecture Documentation"
    author: Red Hat / Debezium Community
    url: https://debezium.io/documentation/reference/stable/architecture.html
    type: official_docs
    published: 2025-12-16
    reliability: authoritative
  - id: src2
    title: "Oracle GoldenGate Licensing Overview"
    author: Redress Compliance
    url: https://redresscompliance.com/oracle-goldengate-licensing-overview/
    type: technical_blog
    published: 2025-01-15
    reliability: moderate_high
  - id: src3
    title: "Oracle CDC: Methods, Benefits, Challenges"
    author: Striim
    url: https://www.striim.com/blog/oracle-cdc/
    type: technical_blog
    published: 2025-03-10
    reliability: moderate_high
  - id: src4
    title: "SAP CDC Architecture — Azure Data Factory"
    author: Microsoft
    url: https://learn.microsoft.com/en-us/azure/data-factory/sap-change-data-capture-introduction-architecture
    type: official_docs
    published: 2025-02-13
    reliability: authoritative
  - id: src5
    title: "Salesforce Change Data Capture Developer Guide v66.0"
    author: Salesforce
    url: https://developer.salesforce.com/docs/atlas.en-us.change_data_capture.meta/change_data_capture/cdc_allocations.htm
    type: official_docs
    published: 2026-02-01
    reliability: authoritative
  - id: src6
    title: "CDC Methods — What It Is and How It Works"
    author: Striim
    url: https://www.striim.com/blog/change-data-capture-cdc-what-it-is-and-how-it-works/
    type: technical_blog
    published: 2025-06-20
    reliability: moderate_high
  - id: src7
    title: "Debezium 3.4.0.Final Release Notes"
    author: Debezium Community
    url: https://debezium.io/blog/2025/12/16/debezium-3-4-final-released/
    type: vendor_release_notes
    published: 2025-12-16
    reliability: high
---

# Change Data Capture (CDC) for ERP Integration — Debezium, GoldenGate, and Cloud ERP Patterns

## TL;DR

- **Bottom line**: Log-based CDC (Debezium, GoldenGate) is the gold standard for on-premise ERP databases — reads transaction logs with near-zero impact. Cloud SaaS ERPs (Salesforce, Workday, NetSuite) block log access entirely; you must use vendor-native event streams or API-based CDC instead.
- **Key limit**: Cloud ERPs cannot do log-based CDC — Salesforce retains CDC events for only 3 days, SAP requires SLT proxy for table-level CDC, and NetSuite/Workday offer no native CDC at all.
- **Watch out for**: Attempting log-based CDC on a SaaS ERP you do not control — it is architecturally impossible. Also, GoldenGate licensing ($17,500/processor on both source AND target) catches teams off guard.
- **Best for**: Real-time, event-driven replication from ERP databases where sub-second latency and zero-loss delivery matter more than implementation simplicity.
- **Authentication**: N/A at CDC level — Debezium uses DB credentials (replication slots for Postgres, binlog reader for MySQL, LogMiner for Oracle); GoldenGate uses OS-level DB access; Salesforce CDC uses OAuth 2.0 via Pub/Sub API.

## System Profile

This card covers Change Data Capture as an integration architecture pattern across multiple ERP systems. It compares log-based CDC tools (Debezium, Oracle GoldenGate), vendor-native CDC (Salesforce CDC, SAP ODP/SLT), and explains why cloud SaaS ERPs fundamentally limit what CDC methods are available. This is not a single-system card — it is a cross-system architecture pattern reference.

The critical insight: CDC method availability depends entirely on whether you have database-level access. On-premise ERPs with relational databases (Oracle EBS, SAP ECC on HANA/Oracle/SQL Server, custom ERPs on PostgreSQL/MySQL) support log-based CDC. Cloud SaaS ERPs (Salesforce, Workday, NetSuite) do not expose transaction logs and require completely different approaches.

| System | CDC Method Available | Tool | DB Access Required? | Latency |
|---|---|---|---|---|
| **Oracle EBS** (on-prem) | Log-based (redo logs) | GoldenGate, Debezium | Yes — supplemental logging | Sub-second |
| **SAP S/4HANA** (on-prem) | Log-based + ODP | SLT, Debezium (for underlying DB), ADF SAP CDC connector | Yes (DB) or No (ODP via RFC) | Seconds |
| **SAP S/4HANA Cloud** | ODP only (no DB log access) | Azure Data Factory SAP CDC, SAP DI | No — ODP/RFC only | Seconds to minutes |
| **Salesforce** | Vendor-native event stream | Salesforce CDC (Pub/Sub API) | No — no DB access | Sub-second (within SF) |
| **NetSuite** | None native; API polling only | Custom (SuiteTalk/REST) | No — no DB access | Minutes (polling interval) |
| **Workday** | None native; API-based only | Workday RaaS / Integration Cloud | No — no DB access | Minutes to hours |
| **Dynamics 365** | Dataverse Change Tracking | Dataverse API / Azure Synapse Link | No — API-level only | Minutes |
| **Custom ERP (PostgreSQL)** | Log-based (WAL) | Debezium | Yes — logical replication | Sub-second |
| **Custom ERP (MySQL)** | Log-based (binlog) | Debezium | Yes — binlog access | Sub-second |

## API Surfaces & Capabilities

| CDC Tool/Method | Protocol | Best For | Max Throughput | Latency | Open Source? | Cost |
|---|---|---|---|---|---|---|
| **Debezium** (Kafka Connect) | Kafka protocol / HTTP (Server) | On-prem DB CDC at scale | 100K+ events/sec (cluster dependent) | Sub-second | Yes (Apache 2.0) | Infrastructure only |
| **Oracle GoldenGate** | Trail files / REST API | Oracle-to-Oracle, high-volume Oracle CDC | 100K+ events/sec | Sub-second | No | $17,500/processor |
| **Salesforce CDC** | Pub/Sub gRPC / CometD | Salesforce record change tracking | Edition-dependent daily allocation | Sub-second (within SF) | N/A (platform feature) | Included in Enterprise+ |
| **SAP ODP/SLT** | RFC / OData | SAP table and CDS view extraction | Depends on SLT sizing | Seconds | No | SAP licensing |
| **Azure Data Factory SAP CDC** | ODP via RFC | SAP-to-Azure delta extraction | Depends on IR sizing | Minutes | No | ADF pricing (per pipeline) |
| **Query-based polling** | REST/SOAP API | Simple, low-volume, any ERP | Limited by API rate limits | Minutes to hours | N/A | API call costs |
| **Trigger-based CDC** | DB triggers + shadow tables | Legacy systems, no log access | Low (trigger overhead) | Seconds | N/A | Development cost |

## Rate Limits & Quotas

### Debezium Limits

| Limit Type | Value | Applies To | Notes |
|---|---|---|---|
| Max connectors per Connect cluster | No hard limit | Kafka Connect | Bounded by cluster resources (CPU, memory) |
| Snapshot lock duration | Varies by DB | Initial snapshot | MySQL: global read lock; PostgreSQL: ACCESS SHARE (non-blocking) |
| Kafka message max size | 1 MB default | All connectors | Configure `max.message.bytes` for large rows; consider chunking |
| Heartbeat interval | 300,000 ms default | All connectors | Reduce for low-traffic tables to prevent WAL/binlog retention issues |
| Oracle LogMiner batch size | 10,000 rows default | Oracle connector | `log.mining.batch.size.default`; increase for high-volume |
| Slot replication lag | Monitor required | PostgreSQL | Unbounded WAL growth if connector falls behind — disk exhaustion risk |

[src1]

### Oracle GoldenGate Limits

| Limit Type | Value | Applies To | Notes |
|---|---|---|---|
| Trail file size | 2 GB default (configurable) | Extract/Replicat | Split larger transactions across trail files |
| Concurrent Extract processes | OS/resource bound | Source DB | Each Extract adds redo log read overhead |
| Supplemental logging overhead | 5-15% write increase | Source Oracle DB | Required for log-based capture — cannot be avoided |
| Max transaction size | Limited by trail disk space | Large batch operations | Very large transactions (>10M rows) may require tuning |

[src2, src3]

### Salesforce CDC Limits

| Limit Type | Value | Window | Edition Differences |
|---|---|---|---|
| Event retention | 3 days | Rolling | Same across all editions |
| Max entities per CDC channel | 5 (custom channel) | Per channel | Standard channel covers all entities |
| Daily CDC event delivery allocation | Edition-dependent | 24h rolling | Enterprise: included; check Salesforce Limits Quick Reference |
| Pub/Sub API subscribe requests | Counted against daily API limits | 24h rolling | Concurrent subscriber limit applies |
| CometD streaming connections | Based on edition | Concurrent | Enterprise: 2,000 concurrent clients |

[src5]

### SAP ODP/SLT Limits

| Limit Type | Value | Applies To | Notes |
|---|---|---|---|
| ODP delta queue retention | Configurable (default 24h) | All ODP subscribers | Old deltas purged after all subscribers consume |
| SLT replication tables | OS/DB resource bound | SLT server | Each replicated table requires SLT logging table |
| ADF SAP CDC throughput | Self-hosted IR dependent | Azure Data Factory | Serverless Spark cluster for transformation |

[src4]

## Authentication

| Tool | Auth Method | Credentials | Refresh? | Notes |
|---|---|---|---|---|
| Debezium (PostgreSQL) | DB user with REPLICATION role | Username/password or SSL cert | N/A | Create dedicated replication user with minimal privileges |
| Debezium (MySQL) | DB user with REPLICATION SLAVE, REPLICATION CLIENT | Username/password | N/A | Binlog access; also needs SELECT on captured tables |
| Debezium (Oracle) | DB user with LogMiner privileges | Username/password | N/A | Requires SELECT on V$LOG, V$LOGFILE, V$ARCHIVED_LOG, etc. |
| Oracle GoldenGate | OS-level DB access + GoldenGate admin | DB credentials + GG credential store | N/A | Extract process runs as OS user with DB read access |
| Salesforce CDC | OAuth 2.0 (JWT or Web Server flow) | Connected App + digital certificate | Yes (access token: 2h) | Subscribe via Pub/Sub API (gRPC) or CometD (Bayeux) |
| SAP ODP (via ADF) | SAP RFC user | SAP username/password | N/A | Self-hosted Integration Runtime required for on-prem SAP |

[src1, src4, src5]

### Authentication Gotchas

- **Debezium PostgreSQL**: The replication user must have `REPLICATION` role AND `LOGIN` privilege. It also needs `SELECT` on tables being captured. Logical replication slot is created automatically but persists after connector shutdown — abandoned slots cause WAL disk exhaustion. [src1]
- **Oracle GoldenGate**: Extract processes require supplemental logging to be enabled at both database and table level. Forgetting table-level supplemental logging causes silent data loss — changes are captured but without sufficient column data for downstream apply. [src3]
- **Salesforce CDC**: OAuth tokens expire, and if the Pub/Sub API subscriber disconnects, it can replay events only within the 3-day retention window. After 3 days, events are permanently lost. [src5]

## Constraints
<!-- Agents: read this section before recommending any integration approach from this unit.
     These are hard constraints that must not be violated. -->

- **Cloud SaaS ERPs block log-based CDC entirely**: Salesforce, Workday, NetSuite, and other SaaS ERPs do not expose database transaction logs. Do not recommend Debezium or GoldenGate for these systems — it is architecturally impossible.
- **GoldenGate dual-processor licensing**: License is required on BOTH source and target processors. A 4-processor source + 4-processor target = 8 processor licenses = $140,000 list price before support. [src2]
- **Salesforce CDC 3-day retention**: Events not consumed within 3 days are permanently deleted. Any subscriber downtime > 3 days causes data loss requiring full re-sync. [src5]
- **Debezium requires Kafka or Debezium Server**: Cannot run log-based CDC without message broker infrastructure. Minimum viable Kafka cluster: 3 brokers + KRaft (or ZooKeeper). Debezium Server is lighter but still needs a sink (Kinesis, Pub/Sub, HTTP). [src1]
- **Oracle supplemental logging overhead**: Enabling supplemental logging (required for both GoldenGate and Debezium Oracle connector) increases redo log volume by 5-15%. On high-write Oracle databases, this impacts I/O and archive log storage. [src3]
- **SAP SLT licensing**: SLT replication server is a separate SAP product requiring its own license. Not included in base S/4HANA — adds infrastructure and cost for table-level CDC. [src4]
- **PostgreSQL WAL retention risk**: Debezium creates a replication slot that prevents WAL cleanup. If the connector falls behind or crashes without cleanup, WAL files grow unbounded until disk is full — causing database outage. [src1]

## Integration Pattern Decision Tree

```
START — Need CDC from an ERP system
├── Do you have direct database access?
│   ├── YES (on-premise / IaaS with DB admin) ──────────────────────────────┐
│   │   ├── Which database engine?                                          │
│   │   │   ├── Oracle → Debezium (LogMiner) or GoldenGate                 │
│   │   │   │   ├── Budget allows $17,500+/processor?                       │
│   │   │   │   │   ├── YES + Oracle-to-Oracle → GoldenGate (best Oracle)  │
│   │   │   │   │   └── NO or heterogeneous targets → Debezium (free, Kafka)│
│   │   │   ├── PostgreSQL → Debezium (logical replication, WAL)            │
│   │   │   ├── MySQL/MariaDB → Debezium (binlog)                          │
│   │   │   ├── SQL Server → Debezium (SQL Server CDC feature)             │
│   │   │   └── Db2 → Debezium (Db2 connector)                             │
│   │   └── Data volume?                                                    │
│   │       ├── < 10K changes/day → Debezium single-node may suffice       │
│   │       ├── 10K-1M changes/day → Debezium on 3-node Kafka cluster      │
│   │       └── > 1M changes/day → Debezium on scaled Kafka + partitioning │
│   └── NO (SaaS / cloud ERP, no DB access) ──────────────────────────────┐
│       ├── Which SaaS ERP?                                                │
│       │   ├── Salesforce → Salesforce CDC (Pub/Sub API)                  │
│       │   │   └── Need >3 day replay? → Add external event store (Kafka) │
│       │   ├── SAP S/4HANA Cloud → ODP via ADF SAP CDC connector          │
│       │   ├── SAP ECC → ODP via SLT (requires SLT license)              │
│       │   ├── NetSuite → API polling (no native CDC)                     │
│       │   │   └── Use SuiteQL + SystemModstamp filter for delta          │
│       │   ├── Workday → RaaS polling or Integration Cloud                │
│       │   └── Dynamics 365 → Dataverse Change Tracking API               │
│       └── Latency requirement?                                           │
│           ├── Sub-second → Only Salesforce CDC delivers this natively    │
│           ├── Seconds → SAP ODP/SLT can achieve this                     │
│           └── Minutes → API polling is acceptable                        │
└── Error tolerance?
    ├── Zero-loss required → Exactly-once (Debezium 3.3+ supports this)
    │   └── Also add dead letter queue for poison messages
    └── At-least-once acceptable → Default Debezium / GoldenGate behavior
```

## Quick Reference

### CDC Method Comparison

| Method | Mechanism | Latency | DB Impact | Deletes Captured? | Schema Changes? | Complexity | Cost |
|---|---|---|---|---|---|---|---|
| **Log-based (Debezium)** | Reads transaction log (WAL/binlog/redo) | Sub-second | Minimal (log tailing) | Yes | Yes (with SMT) | Medium-High | Free + Kafka infra |
| **Log-based (GoldenGate)** | Reads Oracle redo logs | Sub-second | Minimal (log mining) | Yes | Yes | High | $17,500/processor |
| **Vendor-native (SF CDC)** | Platform event bus | Sub-second | None (SaaS managed) | Yes | Partial | Low | Included in Enterprise+ |
| **Vendor-native (SAP ODP)** | ODP delta queue | Seconds | Low (application layer) | Yes | CDS-dependent | Medium | SAP + SLT license |
| **Query-based polling** | Timestamp/ID column filter | Minutes | High (repeated queries) | NO | N/A | Low | API call costs |
| **Trigger-based** | DB triggers write to shadow table | Seconds | HIGH (per-transaction) | Yes | Manual | Medium | Development cost |

[src1, src3, src6]

### Tool Selection Matrix

| Factor | Debezium | GoldenGate | Salesforce CDC | SAP ODP/SLT | Polling |
|---|---|---|---|---|---|
| **License cost** | Free (Apache 2.0) | $17,500/processor | Included | SAP license | Free |
| **Infrastructure** | Kafka cluster required | GoldenGate hub | None (SaaS) | SLT server (table CDC) | None |
| **Supported sources** | 11 databases | Oracle primary + others | Salesforce only | SAP only | Any with API |
| **Exactly-once** | Yes (v3.3+) | Yes (with bounded recovery) | At-most-once (no replay guarantee) | At-least-once | N/A |
| **Schema evolution** | Auto-detect, configurable | Supports DDL replication | Automatic | CDS view dependent | N/A |
| **Operational complexity** | Medium (Kafka ops) | High (GG admin) | Low | Medium (SLT admin) | Low |
| **Community/support** | Large open-source community | Oracle support (paid) | Salesforce support | SAP support | N/A |

## Step-by-Step Integration Guide

### 1. Set up Debezium for PostgreSQL-backed ERP

Deploy Debezium via Kafka Connect to capture changes from a PostgreSQL-backed ERP database. This is the most common open-source CDC pattern. [src1]

```bash
# Prerequisites: Kafka cluster running, PostgreSQL with wal_level=logical

# 1. Enable logical replication on PostgreSQL
# In postgresql.conf:
#   wal_level = logical
#   max_replication_slots = 4
#   max_wal_senders = 4

# 2. Create a dedicated replication user
psql -c "CREATE ROLE debezium_user WITH REPLICATION LOGIN PASSWORD 'secure_password';"
psql -c "GRANT SELECT ON ALL TABLES IN SCHEMA public TO debezium_user;"
```

**Verify**: `psql -c "SHOW wal_level;"` --> expected: `logical`

### 2. Deploy Debezium PostgreSQL connector

Register the Debezium connector with Kafka Connect. [src1]

```bash
# Register Debezium PostgreSQL connector via Kafka Connect REST API
curl -X POST http://localhost:8083/connectors \
  -H "Content-Type: application/json" \
  -d '{
    "name": "erp-postgres-cdc",
    "config": {
      "connector.class": "io.debezium.connector.postgresql.PostgresConnector",
      "database.hostname": "erp-db.internal",
      "database.port": "5432",
      "database.user": "debezium_user",
      "database.password": "secure_password",
      "database.dbname": "erp_production",
      "topic.prefix": "erp",
      "schema.include.list": "public",
      "table.include.list": "public.orders,public.customers,public.invoices",
      "slot.name": "debezium_erp",
      "plugin.name": "pgoutput",
      "publication.autocreate.mode": "filtered",
      "snapshot.mode": "initial",
      "heartbeat.interval.ms": "30000",
      "tombstones.on.delete": "true"
    }
  }'
```

**Verify**: `curl http://localhost:8083/connectors/erp-postgres-cdc/status` --> expected: `"state":"RUNNING"`

### 3. Consume CDC events from Kafka

Read change events from the Kafka topic created by Debezium. [src1]

```bash
# List topics created by Debezium
kafka-topics.sh --bootstrap-server localhost:9092 --list | grep erp

# Consume change events from the orders table topic
kafka-console-consumer.sh \
  --bootstrap-server localhost:9092 \
  --topic erp.public.orders \
  --from-beginning \
  --max-messages 5
```

**Verify**: Output should contain JSON with `"op": "c"` (create), `"op": "u"` (update), or `"op": "d"` (delete) fields.

### 4. Monitor replication slot health

Prevent WAL disk exhaustion by monitoring replication slot lag. [src1]

```sql
-- Check replication slot status and lag
SELECT slot_name,
       active,
       pg_size_pretty(pg_wal_lsn_diff(pg_current_wal_lsn(), restart_lsn)) AS lag_size,
       pg_size_pretty(pg_wal_lsn_diff(pg_current_wal_lsn(), confirmed_flush_lsn)) AS pending_size
FROM pg_replication_slots
WHERE slot_name = 'debezium_erp';
```

**Verify**: `lag_size` should be < 1 GB under normal operation. If > 5 GB, investigate connector health immediately.

## Code Examples

### Python: Subscribe to Salesforce CDC events via Pub/Sub API

```python
# Input:  Salesforce Connected App credentials (client_id, client_secret, username)
# Output: Stream of change events for subscribed Salesforce objects

# pip install grpcio==1.62.0 grpcio-tools==1.62.0 requests==2.31.0

import grpc
import requests
import json

# Step 1: Authenticate via OAuth 2.0
def get_salesforce_token(client_id, client_secret, username, password, security_token):
    """Authenticate with Salesforce and return access token + instance URL."""
    resp = requests.post("https://login.salesforce.com/services/oauth2/token", data={
        "grant_type": "password",
        "client_id": client_id,
        "client_secret": client_secret,
        "username": username,
        "password": password + security_token
    })
    resp.raise_for_status()
    data = resp.json()
    return data["access_token"], data["instance_url"]

# Step 2: Subscribe to CDC channel via Pub/Sub API (gRPC)
# Note: Salesforce Pub/Sub API requires proto file from Salesforce
# See: https://developer.salesforce.com/docs/platform/pub-sub-api/
def subscribe_to_cdc(access_token, instance_url, channel="/data/ChangeEvents"):
    """Subscribe to Salesforce CDC events. Events retained for 3 days only."""
    # gRPC channel setup (simplified — production needs proper proto compilation)
    print(f"Subscribing to {channel} on {instance_url}")
    print("WARNING: Events retained for 3 days only — ensure consumer uptime")
    # In production: use grpc.secure_channel with Salesforce Pub/Sub API endpoint
    # Process events and persist to durable store (Kafka, database) immediately

# Step 3: Process change events
def process_change_event(event):
    """Extract operation type and changed fields from CDC event."""
    header = event.get("ChangeEventHeader", {})
    return {
        "entity": header.get("entityName"),
        "operation": header.get("changeType"),  # CREATE, UPDATE, DELETE, UNDELETE
        "record_ids": header.get("recordIds"),
        "changed_fields": header.get("changedFields"),
        "commit_timestamp": header.get("commitTimestamp"),
        "transaction_key": header.get("transactionKey"),
    }
```

### JavaScript/Node.js: Debezium CDC event consumer via Kafka

```javascript
// Input:  Kafka cluster connection, Debezium topic name
// Output: Processed change events with operation type and payload

// npm install kafkajs@2.2.4

const { Kafka } = require('kafkajs');

const kafka = new Kafka({
  clientId: 'erp-cdc-consumer',
  brokers: ['kafka-1:9092', 'kafka-2:9092', 'kafka-3:9092'],
});

const consumer = kafka.consumer({ groupId: 'erp-sync-group' });

async function consumeCDCEvents(topicPattern = /^erp\.public\./) {
  await consumer.connect();
  await consumer.subscribe({ topics: topicPattern, fromBeginning: false });

  await consumer.run({
    eachMessage: async ({ topic, partition, message }) => {
      const key = message.key ? JSON.parse(message.key.toString()) : null;
      const value = message.value ? JSON.parse(message.value.toString()) : null;

      if (!value) {
        // Tombstone event — record was deleted (if tombstones.on.delete=true)
        console.log(`DELETE detected for key: ${JSON.stringify(key)}`);
        return;
      }

      const { op, before, after, source } = value;
      // op: 'c' = create, 'u' = update, 'd' = delete, 'r' = read (snapshot)
      console.log(`[${source.table}] op=${op} ts=${source.ts_ms}`);

      switch (op) {
        case 'c': // INSERT
        case 'r': // Snapshot read (initial load)
          await upsertToTarget(source.table, after);
          break;
        case 'u': // UPDATE
          await upsertToTarget(source.table, after);
          break;
        case 'd': // DELETE
          await deleteFromTarget(source.table, before);
          break;
      }
    },
  });
}

async function upsertToTarget(table, record) {
  // Implement: write to target system (data warehouse, cache, search index)
  console.log(`Upsert ${table}:`, record);
}

async function deleteFromTarget(table, record) {
  // Implement: delete from target system
  console.log(`Delete ${table}:`, record);
}

consumeCDCEvents().catch(console.error);
```

### cURL: Check Debezium connector status and metrics

```bash
# Input:  Kafka Connect REST API endpoint
# Output: Connector status, task states, and lag metrics

# List all deployed connectors
curl -s http://localhost:8083/connectors | jq .

# Check specific connector status
curl -s http://localhost:8083/connectors/erp-postgres-cdc/status | jq .
# Expected: {"name":"erp-postgres-cdc","connector":{"state":"RUNNING"},"tasks":[{"state":"RUNNING"}]}

# Get connector configuration
curl -s http://localhost:8083/connectors/erp-postgres-cdc/config | jq .

# Pause connector (for maintenance)
curl -X PUT http://localhost:8083/connectors/erp-postgres-cdc/pause

# Resume connector
curl -X PUT http://localhost:8083/connectors/erp-postgres-cdc/resume

# Delete connector (WARNING: does NOT drop the replication slot — clean up manually)
curl -X DELETE http://localhost:8083/connectors/erp-postgres-cdc
# IMPORTANT: After deleting, manually drop the replication slot:
# psql -c "SELECT pg_drop_replication_slot('debezium_erp');"
```

## Data Mapping

### CDC Event Structure (Debezium Envelope)

| Field | Type | Description | Gotcha |
|---|---|---|---|
| `op` | String (c/u/d/r) | Operation type: create, update, delete, read (snapshot) | Snapshot events use `r`, not `c` — handle both in consumer logic |
| `before` | Object | Record state BEFORE change (null for inserts) | Only populated if `REPLICA IDENTITY FULL` set on PostgreSQL table |
| `after` | Object | Record state AFTER change (null for deletes) | This is the payload you sync to target systems |
| `source.ts_ms` | Long | Source database timestamp of the change | Use this for ordering, NOT Kafka message timestamp |
| `source.lsn` | Long | Log Sequence Number (PostgreSQL) / binlog position (MySQL) | Monotonically increasing — use for deduplication |
| `source.txId` | Long | Transaction ID | Multiple events can share the same txId (multi-row transaction) |
| `ts_ms` | Long | Debezium processing timestamp | Always later than `source.ts_ms` — the delta is your replication lag |

### Salesforce CDC Event Structure

| Field | Type | Description | Gotcha |
|---|---|---|---|
| `ChangeEventHeader.entityName` | String | SObject name (e.g., Account, Opportunity) | Use this to route to correct target table |
| `ChangeEventHeader.changeType` | String | CREATE, UPDATE, DELETE, UNDELETE | UNDELETE is Salesforce-specific — handle it as upsert |
| `ChangeEventHeader.recordIds` | String[] | Affected record IDs (up to 10 per event) | Batched events — one event can contain multiple record IDs |
| `ChangeEventHeader.changedFields` | String[] | Field API names that changed | Only present for UPDATE — use for selective sync |
| `ChangeEventHeader.commitTimestamp` | Long | Timestamp of the data change | Epoch milliseconds — convert for human-readable logging |

### Data Type Gotchas

- **Debezium decimal handling**: Debezium serializes `DECIMAL`/`NUMERIC` types as `bytes` (Avro) or base64-encoded strings (JSON) by default. Configure `decimal.handling.mode=string` or `decimal.handling.mode=double` to avoid downstream parsing headaches. [src1]
- **Timestamp timezone mismatch**: Debezium emits timestamps in UTC by default. Oracle databases may store in local timezone. MySQL `TIMESTAMP` columns auto-convert; `DATETIME` columns do not. Verify `database.time.zone` configuration. [src1]
- **Salesforce compound fields**: Address and geolocation compound fields emit as nested objects in CDC events, not flat fields. This breaks naive flat-table mapping. [src5]
- **NULL semantics differ**: In Debezium, a NULL `before` field means the record was created (no prior state). In Salesforce CDC, `changedFields` omits fields that were not changed — absence means "no change", not "null". [src1, src5]

## Error Handling & Failure Points

### Common Error Codes

| Error | Source | Cause | Resolution |
|---|---|---|---|
| `PSQLException: replication slot already exists` | Debezium/PostgreSQL | Connector re-created without dropping old slot | `SELECT pg_drop_replication_slot('slot_name');` then restart |
| `ORA-01291: missing logfile` | Debezium/Oracle or GoldenGate | Archive log was purged before CDC could read it | Increase archive log retention; reduce mining lag |
| `INSUFFICIENT_ACCESS` | Salesforce CDC | OAuth user lacks CDC permissions | Assign "Subscribe to CDC Events" permission set |
| `ConnectException: Task is in FAILED state` | Debezium/Kafka Connect | Connector task crashed (schema change, network, etc.) | Check `GET /connectors/{name}/status`; restart task; check dead letter queue |
| `WAL segment has already been removed` | Debezium/PostgreSQL | Replication slot fell too far behind | Full re-snapshot required (`snapshot.mode=always` then revert) |
| `GG-01031: Extract abended` | GoldenGate | Extract process crashed (redo log gap, tablespace issue) | Check GoldenGate error log; `START EXTRACT` after fixing root cause |

[src1, src3, src5]

### Failure Points in Production

- **PostgreSQL WAL disk exhaustion**: Debezium replication slot prevents WAL cleanup. If connector is down for hours/days, WAL fills disk, crashing the database. Fix: `Monitor pg_replication_slots.restart_lsn lag; set max_slot_wal_keep_size (PG 13+) to cap WAL retention; alert at 50% disk.` [src1]
- **Oracle supplemental logging disabled on new table**: Adding a new table to CDC without enabling supplemental logging causes silent data corruption — updates captured without full row context. Fix: `ALTER TABLE new_table ADD SUPPLEMENTAL LOG DATA (ALL) COLUMNS; always script this alongside DDL.` [src3]
- **Salesforce subscriber falls behind 3-day window**: Downtime > 3 days means lost events with no way to replay. Fix: `Persist CDC events to durable store (Kafka) immediately; implement full re-sync triggered by gap detection; use replayId tracking.` [src5]
- **Debezium schema change breaks Avro serialization**: Adding a non-nullable column without default in source DB causes Avro schema compatibility failure. Fix: `Use BACKWARD compatible schema registry settings; always add new columns as nullable or with defaults.` [src1]
- **GoldenGate trail file disk full**: Trail files accumulate when Replicat falls behind Extract. Fix: `Monitor trail lag (INFO EXTRACT, DETAIL); set PURGEOLDEXTRACTS to auto-clean; alert when trail lag > 1 hour.` [src3]
- **SAP ODP delta queue overflow**: If subscriber (ADF) does not consume deltas in time, SAP purges the queue. Fix: `Configure ODP queue retention (transaction ODQMON); schedule ADF pipeline runs more frequently than retention window.` [src4]

## Anti-Patterns

### Wrong: Polling an ERP database on a timer for changes

```python
# BAD — Constant load on DB, misses deletes, timestamp gaps lose data
import schedule, psycopg2

def poll_for_changes():
    conn = psycopg2.connect("host=erp-db dbname=erp_prod")
    cur = conn.cursor()
    # This query runs every 60 seconds against production
    cur.execute("SELECT * FROM orders WHERE updated_at > NOW() - INTERVAL '60 seconds'")
    # Problem 1: Misses deletes entirely
    # Problem 2: Clock skew between app and DB causes data loss
    # Problem 3: Adds constant query load to production DB
    # Problem 4: 60-second window means up to 60s latency
    rows = cur.fetchall()
    for row in rows:
        sync_to_target(row)

schedule.every(60).seconds.do(poll_for_changes)
```

### Correct: Log-based CDC with Debezium

```python
# GOOD — Zero query load on DB, captures deletes, sub-second latency
# Debezium reads the WAL/binlog — no queries against production tables
# See the Step-by-Step guide above for full Debezium setup
# Consumer code reads from Kafka topic, not from the database
from kafka import KafkaConsumer  # kafka-python==2.0.2
import json

consumer = KafkaConsumer(
    'erp.public.orders',
    bootstrap_servers=['kafka:9092'],
    value_deserializer=lambda m: json.loads(m.decode('utf-8')),
    group_id='erp-sync',
    auto_offset_reset='earliest',
)
for message in consumer:
    event = message.value
    op = event['op']  # c=create, u=update, d=delete, r=snapshot
    if op in ('c', 'u', 'r'):
        sync_to_target(event['after'])
    elif op == 'd':
        delete_from_target(event['before'])  # Captures deletes!
```

### Wrong: Using GoldenGate for non-Oracle targets

```text
# BAD — Using Oracle GoldenGate to replicate to PostgreSQL/MySQL targets
# GoldenGate licensing requires $17,500/processor on BOTH source AND target
# For heterogeneous replication, you're paying Oracle license for non-Oracle DB
# Total cost: 8 processors * $17,500 = $140,000 + 22% annual support = $170,800 year 1
```

### Correct: Use Debezium for heterogeneous CDC

```text
# GOOD — Debezium is free (Apache 2.0) and supports 11 source databases
# Oracle source -> Debezium (LogMiner) -> Kafka -> Kafka Connect JDBC Sink -> PostgreSQL
# Total license cost: $0 (infrastructure costs for Kafka cluster only)
# Trade-off: Must manage Kafka infrastructure; consider managed Kafka (Confluent, MSK, Aiven)
```

### Wrong: Assuming Salesforce CDC works like database CDC

```python
# BAD — Treating Salesforce CDC as unlimited, durable event stream
def setup_salesforce_cdc():
    # Assumption: events are retained forever like Kafka
    subscribe_to("/data/AccountChangeEvent")
    # Problem 1: Events retained for 3 days ONLY
    # Problem 2: No guarantee of exactly-once delivery
    # Problem 3: If subscriber is down > 3 days, events are LOST
    # Problem 4: Daily event allocation is edition-limited
```

### Correct: Buffer Salesforce CDC into durable store

```python
# GOOD — Immediately persist SF CDC events to Kafka for durability
def setup_salesforce_cdc_pipeline():
    # Step 1: Subscribe to Salesforce CDC
    events = subscribe_to("/data/AccountChangeEvent")

    # Step 2: Immediately write to Kafka (infinite retention configured)
    for event in events:
        kafka_producer.send("sf-account-changes", event)
        # Now events are durable beyond 3-day SF retention
        # Downstream consumers read from Kafka, not from SF directly

    # Step 3: Track replayId for gap detection
    save_replay_id(events.latest_replay_id)
    # If gap detected (missed events), trigger full Account re-sync
```

## Common Pitfalls

- **Forgetting replication slot cleanup (PostgreSQL)**: Deleting a Debezium connector does NOT drop the replication slot. Orphaned slots prevent WAL cleanup, eventually filling disk and crashing the database. Fix: `Always run SELECT pg_drop_replication_slot('slot_name') after connector deletion. Automate with a monitoring script.` [src1]
- **Initial snapshot overwhelming production**: Debezium's initial snapshot reads all existing data, which can lock tables (MySQL global read lock) or generate massive WAL traffic. Fix: `Use snapshot.mode=initial_only during off-peak hours; for PostgreSQL, use snapshot.mode=exported (non-blocking). For Oracle, schedule during maintenance window.` [src1]
- **Not testing with production data volumes**: CDC works perfectly with 1,000 rows in dev but fails with 100M rows in production due to snapshot timeouts, memory pressure, and topic partition sizing. Fix: `Load-test with production-scale data. Size Kafka partitions based on peak change rate. Monitor JVM heap for connector workers.` [src1]
- **Schema evolution without compatibility checks**: Adding a non-nullable column to source DB breaks downstream Avro deserialization. Fix: `Use Schema Registry with BACKWARD compatibility; always add columns as nullable; run schema compatibility check before DDL changes.` [src1]
- **Ignoring GoldenGate licensing for non-production**: GoldenGate licenses apply to dev, staging, AND production environments. Running unlicensed GoldenGate in non-prod is an audit risk. Fix: `Include all environments in GoldenGate license count or use Debezium for non-production CDC.` [src2]
- **SAP ODP without SLT for application tables**: ODP can extract CDS views and BW extractors directly, but application table extraction requires SLT as a proxy. Attempting direct table CDC without SLT silently fails or returns no deltas. Fix: `Deploy SLT replication server for table-level SAP CDC. For CDS views, direct ODP connection (via ADF or custom) works.` [src4]

## Diagnostic Commands

```bash
# ===== Debezium / Kafka Connect Diagnostics =====

# Check all connector statuses
curl -s http://localhost:8083/connectors | jq -r '.[]' | while read c; do
  echo "=== $c ==="
  curl -s "http://localhost:8083/connectors/$c/status" | jq '{state: .connector.state, tasks: [.tasks[].state]}'
done

# Check PostgreSQL replication slot health
psql -c "SELECT slot_name, active, pg_size_pretty(pg_wal_lsn_diff(pg_current_wal_lsn(), restart_lsn)) AS lag FROM pg_replication_slots;"

# Check Kafka consumer group lag (replication delay)
kafka-consumer-groups.sh --bootstrap-server localhost:9092 --group erp-sync-group --describe

# Check Kafka topic message count
kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list localhost:9092 --topic erp.public.orders --time -1

# ===== Oracle GoldenGate Diagnostics =====

# Check GoldenGate process status
# (Run from GoldenGate admin shell — GGSCI)
# INFO ALL
# INFO EXTRACT ext_erp, DETAIL
# INFO REPLICAT rep_erp, DETAIL
# STATS EXTRACT ext_erp, TOTAL, TABLE *.*

# Check supplemental logging status
# sqlplus: SELECT SUPPLEMENTAL_LOG_DATA_MIN, SUPPLEMENTAL_LOG_DATA_ALL FROM V$DATABASE;

# ===== Salesforce CDC Diagnostics =====

# Check CDC event delivery via REST API
curl -s "https://your-instance.salesforce.com/services/data/v66.0/limits" \
  -H "Authorization: Bearer $ACCESS_TOKEN" | jq '.DailyDeliveredPlatformEvents'

# ===== SAP ODP Diagnostics =====
# Transaction ODQMON — Monitor ODP delta queues
# Transaction SLT_DASHBOARD — Monitor SLT replication status
# Transaction SM21 — SAP system log for replication errors
```

## Version History & Compatibility

| Tool | Version | Release Date | Status | Key Changes |
|---|---|---|---|---|
| Debezium 3.4 | 3.4.0.Final | 2025-12-16 | Current | MariaDB connector GA, CockroachDB incubating |
| Debezium 3.3 | 3.3.0.Final | 2025-10-01 | Supported | Exactly-once semantics for all core connectors |
| Debezium 3.0 | 3.0.0.Final | 2024-10-02 | Supported | Java 17+ required, Kafka 3.x baseline, MySQL 9.0 support |
| Debezium 2.7 | 2.7.x | 2024-06 | EOL | Last Java 11 compatible version |
| GoldenGate 23ai | 23ai | 2024 | Current | Microservices architecture, REST API management |
| GoldenGate 21c | 21c | 2021 | Supported | Classic + Microservices; on-prem + OCI |
| GoldenGate 19c | 19c | 2019 | EOL | Last classic-only version |
| Salesforce CDC | API v66.0 | 2026-02 (Spring '26) | Current | Pub/Sub API preferred; CometD still supported |
| ADF SAP CDC | 2025-02 | 2025-02 | GA | ODP framework, CDS views, SLT integration |

[src1, src3, src4, src5, src7]

### Deprecation Policy

**Debezium**: Major versions (2.x to 3.x) may include breaking changes. Minor versions are backward compatible. EOL policy: previous major version receives critical fixes for approximately 6 months after new major release. **Oracle GoldenGate**: Follows Oracle's lifetime support policy — premier support for 5 years, extended support for 3 additional years. **Salesforce CDC**: Tied to Salesforce API versioning — minimum supported API version retires approximately 3 years after release. [src1, src7]

## When to Use / When Not to Use

| Use When | Don't Use When | Use Instead |
|---|---|---|
| Real-time replication from on-prem ERP with DB access | SaaS ERP with no database access (Workday, NetSuite) | API polling with timestamp filters |
| Need to capture all DML including DELETEs | Only need INSERT/UPDATE tracking | Timestamp-based polling (simpler) |
| High change volume (>10K changes/day) | Low volume (<100 changes/day) | Simple REST API polling (less infra) |
| Event-driven architecture with Kafka already deployed | No Kafka infrastructure and no budget to deploy it | Debezium Server to Kinesis/Pub/Sub, or API polling |
| Oracle-to-Oracle homogeneous replication with Oracle support needed | Heterogeneous targets (Oracle to Postgres/MySQL) | Debezium (free, multi-target via Kafka Connect sinks) |
| Salesforce record change tracking with sub-second latency | Need Salesforce CDC with >3 day replay window | Buffer SF CDC to Kafka, then consume from Kafka |
| SAP delta extraction for data warehouse / data lake | SAP real-time event notifications (not data extraction) | SAP Business Events or custom ABAP publishing |

## Cross-System Comparison

| Capability | Debezium | Oracle GoldenGate | Salesforce CDC | SAP ODP/SLT | API Polling |
|---|---|---|---|---|---|
| **CDC Method** | Log-based (WAL/binlog/redo) | Log-based (redo) | Vendor event stream | Application-layer delta | Query-based |
| **Latency** | Sub-second | Sub-second | Sub-second | Seconds | Minutes |
| **Captures Deletes** | Yes | Yes | Yes | Yes | No |
| **Schema Evolution** | Auto-detect + SMT | DDL replication | Automatic | CDS-dependent | N/A |
| **Delivery Guarantee** | At-least-once (exactly-once in 3.3+) | At-least-once | At-most-once | At-least-once | Best-effort |
| **Source DB Impact** | Minimal (log tailing) | Minimal (log mining) | None (SaaS) | Low (app-layer) | High (repeated queries) |
| **License Cost** | Free (Apache 2.0) | $17,500/processor | Included in Enterprise+ | SAP + SLT license | Free |
| **Supported Sources** | 11 databases | Oracle primary | Salesforce only | SAP only | Any with API |
| **Infrastructure Required** | Kafka cluster | GoldenGate hub servers | None | SLT server (table CDC) | None |
| **Event Retention** | Kafka retention (configurable, infinite possible) | Trail files (disk-bound) | 3 days only | ODP queue (configurable) | N/A |
| **Exactly-Once** | Yes (v3.3+) | Bounded recovery | No | No | N/A |
| **Multi-Target Fan-Out** | Native (Kafka topics) | Limited (one Replicat per target) | Limited (subscribers) | One subscriber per queue | N/A |

## Important Caveats

- **Cloud ERP CDC gap is widening**: More organizations migrate to SaaS ERPs (Salesforce, Workday, NetSuite) where log-based CDC is impossible. This forces adoption of vendor-specific event APIs or polling — each with unique limitations and no standardized interface.
- **Debezium is not zero-ops**: Despite being "just a connector," Debezium requires Kafka cluster management, Schema Registry, connector monitoring, replication slot cleanup, and snapshot planning. Managed Kafka services (Confluent Cloud, MSK, Aiven) reduce but do not eliminate operational burden.
- **GoldenGate cost is often underestimated**: Dual-processor licensing, annual 22% support fees, and requirement to license non-production environments can triple the expected cost. Always get a formal Oracle quote before committing.
- **Exactly-once is still young**: Debezium 3.3+ exactly-once semantics require Kafka transactions support on both producer and consumer sides. Most sink connectors do not yet support transactional consumption — test end-to-end before claiming exactly-once.
- **CDC is not a backup**: CDC streams forward from a point in time. It does not replace database backups, point-in-time recovery, or disaster recovery strategies. Losing the replication slot or Kafka topic means a full re-snapshot is required.

## Related Units

- [Salesforce Platform Events and CDC](/business/erp-integration/salesforce-platform-events/2026)
- [SAP CDC with Azure Data Factory](/business/erp-integration/sap-azure-data-factory-cdc/2026)
- [Real-time ERP Synchronization Patterns](/business/erp-integration/erp-real-time-sync-patterns/2026)
- [API Polling for ERP Integration](/business/erp-integration/erp-api-polling-patterns/2026)