--- # === IDENTITY === id: software/devops/docker-kafka-zookeeper/2026 canonical_question: "Docker Compose reference: Kafka + Zookeeper" aliases: - "docker compose kafka zookeeper setup" - "kafka docker compose yml example" - "confluent kafka docker development environment" - "kafka kraft mode docker compose" - "docker compose kafka schema registry" - "kafka zookeeper docker local development" - "how to run kafka in docker" - "kafka docker compose with UI" entity_type: software_reference domain: software > devops > Docker Kafka Zookeeper region: global jurisdiction: global temporal_scope: 2024-2026 # === VERIFICATION === last_verified: 2026-02-27 confidence: 0.94 version: 1.0 first_published: 2026-02-27 # === TEMPORAL VALIDITY === temporal_validity: status: stable last_breaking_change: "Kafka 4.0 removed ZooKeeper support entirely (2025); KRaft production-ready since Kafka 3.3.1 (2022)" next_review: 2026-08-26 change_sensitivity: medium # === CONSTRAINTS === constraints: - "Kafka 4.0+ does NOT support ZooKeeper -- use KRaft mode exclusively for new clusters" - "KAFKA_ADVERTISED_LISTENERS must reflect the address clients use to connect, not the internal container hostname" - "Single-broker dev clusters MUST set KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR=1 (default 3 causes under-replicated partitions)" - "All brokers in a KRaft cluster MUST share the same CLUSTER_ID -- generate once with kafka-storage random-uuid" - "Never use confluentinc/cp-kafka in production without understanding Confluent licensing terms" # === SKIP CONDITIONS === skip_this_unit_if: - condition: "Need managed Kafka (Confluent Cloud, AWS MSK, Redpanda Cloud)" use_instead: "Managed service documentation for your provider" - condition: "Need Kubernetes deployment (Strimzi, Confluent for Kubernetes)" use_instead: "software/devops/kafka-kubernetes/2026" - condition: "Need Kafka Streams or ksqlDB application development" use_instead: "software/data/kafka-streams/2026" # === AGENT HINTS === inputs_needed: - key: "kafka_version" question: "Which Kafka version are you targeting?" type: choice options: ["Kafka 4.0+ (KRaft only)", "Kafka 3.x (KRaft or ZooKeeper)", "Kafka 2.x (ZooKeeper only)"] - key: "use_case" question: "What is this for?" type: choice options: ["Local development", "CI/CD testing", "Multi-broker staging", "Production"] - key: "extras" question: "Which additional services do you need?" type: choice options: ["Broker only", "Schema Registry", "Kafka UI", "Full stack (Schema Registry + Connect + UI)"] # === DISTRIBUTION === canonical_source: "https://knowledgelib.io/software/devops/docker-kafka-zookeeper/2026" suggested_citation: "Source: knowledgelib.io -- AI Knowledge Library (verified 2026-02-27)" # === RELATED UNITS === related_kos: depends_on: [] related_to: - id: "software/devops/docker-compose-reference/2026" label: "Docker Compose Reference" solves: [] alternative_to: [] often_confused_with: [] # === SOURCES (7 authoritative sources) === # Types: official_docs, technical_blog, rfc_spec, academic_paper, community_resource, industry_report # Reliability: high, moderate_high, moderate, moderate_low, low, authoritative sources: - id: src1 title: "Docker Image Configuration Reference for Confluent Platform" author: Confluent url: https://docs.confluent.io/platform/current/installation/docker/config-reference.html type: official_docs published: 2025-12-01 reliability: authoritative - id: src2 title: "Apache Kafka Docker Getting Started" author: Apache Software Foundation url: https://kafka.apache.org/41/getting-started/docker/ type: official_docs published: 2025-06-01 reliability: authoritative - id: src3 title: "KIP-833: Mark KRaft as Production Ready" author: Apache Software Foundation url: https://cwiki.apache.org/confluence/display/KAFKA/KIP-833:+Mark+KRaft+as+Production+Ready type: official_docs published: 2022-10-03 reliability: authoritative - id: src4 title: "Kafka Listeners Explained" author: Confluent url: https://www.confluent.io/blog/kafka-listeners-explained/ type: technical_blog published: 2023-08-15 reliability: high - id: src5 title: "cp-all-in-one Docker Compose files" author: Confluent url: https://github.com/confluentinc/cp-all-in-one type: community_resource published: 2025-11-01 reliability: high - id: src6 title: "Kafka's Shift from ZooKeeper to KRaft" author: Baeldung url: https://www.baeldung.com/kafka-shift-from-zookeeper-to-kraft type: technical_blog published: 2025-03-01 reliability: moderate_high - id: src7 title: "Kafka UI Docker Compose Documentation" author: Kafbat url: https://github.com/kafbat/kafka-ui/blob/main/documentation/compose/DOCKER_COMPOSE.md type: community_resource published: 2025-06-01 reliability: moderate_high --- # Docker Compose Kafka + Zookeeper: Complete Setup Reference ## TL;DR - **Bottom line**: Use KRaft mode (no ZooKeeper) for all new Kafka clusters; ZooKeeper mode is only needed for legacy Kafka 2.x/3.x migrations. - **Key tool/command**: `docker compose up -d` with properly configured `KAFKA_ADVERTISED_LISTENERS` for external client access. - **Watch out for**: Mismatched `KAFKA_ADVERTISED_LISTENERS` (clients get unreachable internal hostname) and forgetting `KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR=1` in single-broker setups. - **Works with**: Confluent images (cp-kafka 7.x), official Apache images (apache/kafka 3.7-4.x), Bitnami images. Docker Compose v2+. ## Constraints - Kafka 4.0+ does NOT support ZooKeeper -- use KRaft mode exclusively for new clusters - KAFKA_ADVERTISED_LISTENERS must reflect the address clients use to connect, not the internal container hostname - Single-broker dev clusters MUST set KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR=1 (default 3 causes under-replicated partitions) - All brokers in a KRaft cluster MUST share the same CLUSTER_ID -- generate once with `kafka-storage random-uuid` - Never use confluentinc/cp-kafka in production without understanding Confluent licensing terms ## Quick Reference | Service | Image | Ports | Volumes | Key Env | |---|---|---|---|---| | Kafka (KRaft) | `confluentinc/cp-kafka:7.7.1` | `9092:9092`, `29092:29092` | `kafka-data:/var/lib/kafka/data` | `KAFKA_PROCESS_ROLES=broker,controller` | | Kafka (KRaft) | `apache/kafka:3.9.1` | `9092:9092` | `kafka-data:/opt/kafka/data` | `KAFKA_NODE_ID=1` | | Kafka (ZooKeeper) | `confluentinc/cp-kafka:7.4.4` | `9092:9092`, `29092:29092` | `kafka-data:/var/lib/kafka/data` | `KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181` | | ZooKeeper | `confluentinc/cp-zookeeper:7.4.4` | `2181:2181` | `zk-data:/var/lib/zookeeper/data` | `ZOOKEEPER_CLIENT_PORT=2181` | | Schema Registry | `confluentinc/cp-schema-registry:7.7.1` | `8081:8081` | -- | `SCHEMA_REGISTRY_KAFKASTORE_BOOTSTRAP_SERVERS=kafka:29092` | | Kafka UI | `ghcr.io/kafbat/kafka-ui:latest` | `8080:8080` | -- | `KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS=kafka:29092` | | Kafka Connect | `confluentinc/cp-kafka-connect:7.7.1` | `8083:8083` | `connect-plugins:/usr/share/java` | `CONNECT_BOOTSTRAP_SERVERS=kafka:29092` | | REST Proxy | `confluentinc/cp-kafka-rest:7.7.1` | `8082:8082` | -- | `KAFKA_REST_BOOTSTRAP_SERVERS=kafka:29092` | **Environment Variable Naming Convention (Confluent images):** | Component | Prefix | Example Property | Environment Variable | |---|---|---|---| | Kafka broker | `KAFKA_` | `advertised.listeners` | `KAFKA_ADVERTISED_LISTENERS` | | ZooKeeper | `ZOOKEEPER_` | `client.port` | `ZOOKEEPER_CLIENT_PORT` | | Schema Registry | `SCHEMA_REGISTRY_` | `kafkastore.bootstrap.servers` | `SCHEMA_REGISTRY_KAFKASTORE_BOOTSTRAP_SERVERS` | | Kafka Connect | `CONNECT_` | `bootstrap.servers` | `CONNECT_BOOTSTRAP_SERVERS` | | REST Proxy | `KAFKA_REST_` | `bootstrap.servers` | `KAFKA_REST_BOOTSTRAP_SERVERS` | **Rule**: Prefix + uppercase + replace `.` with `_`. [src1] ## Decision Tree ``` START: Which Kafka deployment do you need? |-- Kafka version 4.0+? | |-- YES -> Use KRaft mode only (ZooKeeper removed) | +-- NO v |-- Kafka 3.3 - 3.9? | |-- YES -> KRaft recommended (production-ready since 3.3.1) | | Use ZooKeeper only if migrating existing cluster | +-- NO v |-- Kafka 2.x or 3.0-3.2? | |-- YES -> Use ZooKeeper mode (KRaft not production-ready) | +-- NO v +-- Need Schema Registry / Connect / UI? |-- Dev/testing -> Add services to same compose file |-- Staging/prod -> Separate compose files or Kubernetes +-- Schema Registry -> Always co-deploy with Kafka broker ``` ## Step-by-Step Guide ### 1. Create KRaft mode docker-compose.yml (recommended) This is the standard setup for Kafka 3.3+ and the only option for Kafka 4.0+. KRaft eliminates ZooKeeper entirely. [src2] [src3] ```yaml # docker-compose.yml -- Kafka KRaft mode (no ZooKeeper) # Works with: Confluent cp-kafka 7.5+ or apache/kafka 3.7+ services: kafka: image: confluentinc/cp-kafka:7.7.1 hostname: kafka container_name: kafka ports: - "9092:9092" # External client access - "29092:29092" # Internal Docker network - "9101:9101" # JMX (optional) environment: # KRaft settings KAFKA_NODE_ID: 1 KAFKA_PROCESS_ROLES: broker,controller KAFKA_CONTROLLER_QUORUM_VOTERS: 1@kafka:9093 KAFKA_CONTROLLER_LISTENER_NAMES: CONTROLLER CLUSTER_ID: MkU3OEVBNTcwNTJENDM2Qk # Generate: kafka-storage random-uuid # Listener config (critical for client connectivity) KAFKA_LISTENERS: PLAINTEXT://kafka:29092,CONTROLLER://kafka:9093,PLAINTEXT_HOST://0.0.0.0:9092 KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:29092,PLAINTEXT_HOST://localhost:9092 KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT # Single-broker dev settings KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1 KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 0 KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1 KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1 # Performance tuning KAFKA_LOG_DIRS: /tmp/kraft-combined-logs KAFKA_JMX_PORT: 9101 KAFKA_JMX_HOSTNAME: localhost volumes: - kafka-data:/tmp/kraft-combined-logs volumes: kafka-data: ``` **Verify**: `docker compose up -d && docker compose exec kafka kafka-topics --bootstrap-server localhost:9092 --list` --> empty list (no error) ### 2. Create ZooKeeper mode docker-compose.yml (legacy) Use only for Kafka 2.x or migrating existing ZooKeeper-based clusters. [src1] [src5] ```yaml # docker-compose.yml -- Kafka with ZooKeeper (legacy, Kafka < 4.0 only) services: zookeeper: image: confluentinc/cp-zookeeper:7.4.4 hostname: zookeeper container_name: zookeeper ports: - "2181:2181" environment: ZOOKEEPER_CLIENT_PORT: 2181 ZOOKEEPER_TICK_TIME: 2000 volumes: - zk-data:/var/lib/zookeeper/data - zk-log:/var/lib/zookeeper/log kafka: image: confluentinc/cp-kafka:7.4.4 hostname: kafka container_name: kafka depends_on: - zookeeper ports: - "9092:9092" - "29092:29092" environment: KAFKA_BROKER_ID: 1 KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181 # Listener config KAFKA_LISTENERS: PLAINTEXT://kafka:29092,PLAINTEXT_HOST://0.0.0.0:9092 KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:29092,PLAINTEXT_HOST://localhost:9092 KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT # Single-broker dev settings KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1 KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1 KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1 KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 0 volumes: - kafka-data:/var/lib/kafka/data volumes: zk-data: zk-log: kafka-data: ``` **Verify**: `docker compose up -d && docker compose exec kafka kafka-topics --bootstrap-server localhost:9092 --list` --> empty list (no error) ### 3. Add Schema Registry Schema Registry manages Avro, Protobuf, and JSON schemas for Kafka topics. Add to either KRaft or ZooKeeper compose file. [src1] ```yaml schema-registry: image: confluentinc/cp-schema-registry:7.7.1 hostname: schema-registry container_name: schema-registry depends_on: - kafka ports: - "8081:8081" environment: SCHEMA_REGISTRY_HOST_NAME: schema-registry SCHEMA_REGISTRY_KAFKASTORE_BOOTSTRAP_SERVERS: kafka:29092 SCHEMA_REGISTRY_LISTENERS: http://0.0.0.0:8081 ``` **Verify**: `curl -s http://localhost:8081/subjects` --> `[]` ### 4. Add Kafka UI Kafka UI provides a web dashboard for topics, consumers, messages, and cluster monitoring. [src7] ```yaml kafka-ui: image: ghcr.io/kafbat/kafka-ui:latest container_name: kafka-ui depends_on: - kafka ports: - "8080:8080" environment: KAFKA_CLUSTERS_0_NAME: local KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS: kafka:29092 KAFKA_CLUSTERS_0_SCHEMAREGISTRY: http://schema-registry:8081 # Optional ``` **Verify**: Open `http://localhost:8080` --> Kafka UI dashboard with cluster info ### 5. Create topics and test messaging After the cluster is running, create topics and test with the built-in CLI tools. [src2] ```bash # Create a topic docker compose exec kafka kafka-topics \ --bootstrap-server localhost:9092 \ --create \ --topic test-topic \ --partitions 3 \ --replication-factor 1 # Produce messages echo "hello world" | docker compose exec -T kafka kafka-console-producer \ --bootstrap-server localhost:9092 \ --topic test-topic # Consume messages docker compose exec kafka kafka-console-consumer \ --bootstrap-server localhost:9092 \ --topic test-topic \ --from-beginning \ --max-messages 1 ``` **Verify**: Consumer outputs `hello world` ## Code Examples ### Full Stack: KRaft + Schema Registry + Kafka UI > Full script: [kraft-full-stack.yml](scripts/kraft-full-stack.yml) (72 lines) ```yaml # docker-compose.yml -- Full Kafka dev stack (KRaft mode) # Services: Kafka broker, Schema Registry, Kafka UI # Usage: docker compose up -d services: kafka: image: confluentinc/cp-kafka:7.7.1 ports: - "9092:9092" environment: KAFKA_NODE_ID: 1 KAFKA_PROCESS_ROLES: broker,controller # ... (see full script) ``` ### Official Apache Image: Minimal KRaft Setup ```yaml # docker-compose.yml -- Minimal setup with official Apache image # No Confluent dependencies, pure Apache Kafka services: kafka: image: apache/kafka:3.9.1 container_name: kafka ports: - "9092:9092" environment: KAFKA_NODE_ID: 1 KAFKA_PROCESS_ROLES: broker,controller KAFKA_LISTENERS: PLAINTEXT://:9092,CONTROLLER://:9093 KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092 KAFKA_CONTROLLER_LISTENER_NAMES: CONTROLLER KAFKA_CONTROLLER_QUORUM_VOTERS: 1@kafka:9093 KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1 KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1 KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1 CLUSTER_ID: MkU3OEVBNTcwNTJENDM2Qk ``` ### Multi-Broker KRaft Cluster (3 brokers) > Full script: [kraft-multi-broker.yml](scripts/kraft-multi-broker.yml) (95 lines) ```yaml # docker-compose.yml -- 3-broker KRaft cluster for staging/testing # Generate CLUSTER_ID: docker compose exec kafka-1 kafka-storage random-uuid services: kafka-1: image: confluentinc/cp-kafka:7.7.1 environment: KAFKA_NODE_ID: 1 KAFKA_PROCESS_ROLES: broker,controller KAFKA_CONTROLLER_QUORUM_VOTERS: 1@kafka-1:9093,2@kafka-2:9093,3@kafka-3:9093 # ... (see full script) ``` ### Python: Produce and Consume with confluent-kafka ```python # Input: Running Kafka broker at localhost:9092 # Output: Produces and consumes a test message from confluent_kafka import Producer, Consumer # confluent-kafka==2.6.1 import json # Producer producer = Producer({'bootstrap.servers': 'localhost:9092'}) producer.produce('test-topic', key='key1', value=json.dumps({'msg': 'hello'})) producer.flush() # Consumer consumer = Consumer({ 'bootstrap.servers': 'localhost:9092', 'group.id': 'test-group', 'auto.offset.reset': 'earliest' }) consumer.subscribe(['test-topic']) msg = consumer.poll(timeout=10.0) if msg and not msg.error(): print(f"Received: {msg.value().decode('utf-8')}") consumer.close() ``` ## Anti-Patterns ### Wrong: Using localhost as advertised listener inside Docker network ```yaml # BAD -- other containers cannot reach "localhost" environment: KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092 ``` ### Correct: Dual listeners for internal and external access ```yaml # GOOD -- separate listeners for Docker network and host machine environment: KAFKA_LISTENERS: PLAINTEXT://kafka:29092,PLAINTEXT_HOST://0.0.0.0:9092 KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:29092,PLAINTEXT_HOST://localhost:9092 KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT ``` ### Wrong: Default replication factor on single broker ```yaml # BAD -- default replication factor is 3, single broker only has 1 environment: KAFKA_BROKER_ID: 1 # Missing: KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1 # Result: topics fail to create, "not enough replicas" errors ``` ### Correct: Set replication factor to 1 for dev ```yaml # GOOD -- match replication factor to broker count environment: KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1 KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1 KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1 ``` ### Wrong: Mixing KRaft and ZooKeeper config ```yaml # BAD -- cannot use both ZooKeeper and KRaft settings simultaneously environment: KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181 # ZooKeeper mode KAFKA_PROCESS_ROLES: broker,controller # KRaft mode # Result: broker fails to start with conflicting configuration error ``` ### Correct: Choose one mode ```yaml # GOOD (KRaft) -- no ZooKeeper references environment: KAFKA_NODE_ID: 1 KAFKA_PROCESS_ROLES: broker,controller KAFKA_CONTROLLER_QUORUM_VOTERS: 1@kafka:9093 CLUSTER_ID: MkU3OEVBNTcwNTJENDM2Qk # GOOD (ZooKeeper) -- no KRaft references environment: KAFKA_BROKER_ID: 1 KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181 ``` ### Wrong: No health check or dependency ordering ```yaml # BAD -- Schema Registry starts before Kafka is ready services: schema-registry: image: confluentinc/cp-schema-registry:7.7.1 # No depends_on, no health check # Result: Schema Registry crashes on startup, may not auto-recover ``` ### Correct: Use depends_on with health checks ```yaml # GOOD -- wait for Kafka to be healthy before starting dependents services: kafka: image: confluentinc/cp-kafka:7.7.1 healthcheck: test: kafka-topics --bootstrap-server localhost:9092 --list interval: 10s timeout: 5s retries: 5 schema-registry: image: confluentinc/cp-schema-registry:7.7.1 depends_on: kafka: condition: service_healthy ``` ## Common Pitfalls - **Clients get "connection refused" from host**: KAFKA_ADVERTISED_LISTENERS is set to the container hostname (`kafka:29092`) instead of `localhost:9092`. Fix: Add a `PLAINTEXT_HOST://localhost:9092` listener for host access. [src4] - **"Not enough replicas" on topic creation**: Default replication factor is 3 but only 1 broker exists. Fix: Set `KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR=1` and `KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR=1`. [src1] - **Data lost on container restart**: Kafka data directory is not mounted to a volume. Fix: Add `volumes: - kafka-data:/var/lib/kafka/data` (Confluent) or `volumes: - kafka-data:/opt/kafka/data` (Apache). [src2] - **Schema Registry fails on startup**: Starts before Kafka is ready. Fix: Use `depends_on` with `condition: service_healthy` and add a health check to the Kafka service. [src5] - **ZooKeeper ensemble ID conflicts**: Multiple ZooKeeper instances use the same `ZOOKEEPER_SERVER_ID`. Fix: Assign unique IDs (1, 2, 3) to each ZooKeeper node. [src1] - **Port conflicts with other services**: Kafka default port 9092 conflicts with other services on the host. Fix: Map to a different host port: `ports: - "19092:9092"`. [src4] - **KRaft cluster ID mismatch**: Brokers in the same cluster have different `CLUSTER_ID` values. Fix: Generate one UUID with `kafka-storage random-uuid` and use it for all brokers. [src3] - **Old Docker Compose v1 syntax**: Using `docker-compose` (v1, hyphenated) command which is deprecated. Fix: Use `docker compose` (v2, space-separated) which is bundled with Docker Desktop. [src2] ## Diagnostic Commands ```bash # Check if Kafka broker is running and responsive docker compose exec kafka kafka-broker-api-versions --bootstrap-server localhost:9092 # List all topics docker compose exec kafka kafka-topics --bootstrap-server localhost:9092 --list # Describe a specific topic (partitions, replicas, ISR) docker compose exec kafka kafka-topics --bootstrap-server localhost:9092 --describe --topic test-topic # Check consumer group lag docker compose exec kafka kafka-consumer-groups --bootstrap-server localhost:9092 --describe --group my-group # View broker logs for errors docker compose logs kafka --tail 50 # Check cluster metadata (KRaft) docker compose exec kafka kafka-metadata --snapshot /tmp/kraft-combined-logs/__cluster_metadata-0/00000000000000000000.log --cluster-id MkU3OEVBNTcwNTJENDM2Qk # Test Schema Registry connectivity curl -s http://localhost:8081/subjects | python3 -m json.tool # Check ZooKeeper status (ZooKeeper mode only) docker compose exec zookeeper echo ruok | nc localhost 2181 # Check Kafka container resource usage docker stats kafka --no-stream # Verify advertised listeners from client perspective docker compose exec kafka kafka-configs --bootstrap-server localhost:9092 --entity-type brokers --entity-name 1 --describe --all | grep advertised ``` ## Version History & Compatibility | Kafka Version | ZooKeeper Support | KRaft Status | Docker Image | Notes | |---|---|---|---|---| | 4.0+ | Removed | Required (only mode) | `apache/kafka:4.0.0` | ZooKeeper code fully removed | | 3.9.x | Deprecated | Production-ready | `apache/kafka:3.9.1`, `cp-kafka:7.8.x` | Last release with ZK support | | 3.7-3.8 | Deprecated | Production-ready | `apache/kafka:3.8.0`, `cp-kafka:7.7.1` | KRaft recommended for new clusters | | 3.3-3.6 | Supported | Production-ready | `cp-kafka:7.3-7.6` | KRaft GA since 3.3.1 | | 3.0-3.2 | Supported | Preview | `cp-kafka:7.0-7.2` | KRaft not recommended for production | | 2.x | Required | Not available | `cp-kafka:6.x` | Must use ZooKeeper | **Confluent Platform to Kafka version mapping:** CP 7.7.x = Kafka 3.7.x, CP 7.8.x = Kafka 3.8.x. [src6] ## When to Use / When Not to Use | Use When | Don't Use When | Use Instead | |---|---|---| | Local development and testing | Production with 99.99% SLA requirements | Managed Kafka (Confluent Cloud, AWS MSK) | | CI/CD pipeline integration testing | Multi-datacenter replication needed | Kubernetes + Strimzi/Confluent Operator | | Learning Kafka architecture | Need auto-scaling based on load | Serverless Kafka (Confluent, Redpanda) | | Prototyping event-driven architectures | Security compliance requires audit logging | Enterprise Confluent Platform | | Docker-based staging environments | Team lacks Docker/container expertise | Native Kafka install or managed service | ## Important Caveats - Confluent images (cp-kafka) include Confluent-specific features and are under Confluent Community License -- not Apache 2.0. The official `apache/kafka` image is fully Apache 2.0 licensed. - KRaft combined mode (broker + controller in same process) is suitable for development only. Production KRaft deployments should separate controller and broker roles. - Docker Desktop on macOS/Windows adds a VM layer that impacts Kafka I/O performance. For benchmarking, use Linux with native Docker. - The `wurstmeister/kafka` image is unmaintained since 2023. Migrate to `confluentinc/cp-kafka` or `apache/kafka`. - Schema Registry stores schemas in a Kafka topic (`_schemas`). Losing this topic loses all schema history. Back up the topic or use Schema Registry export. - Kafka UI tools (Kafbat kafka-ui, AKHQ, Redpanda Console) are community-maintained and may lag behind Kafka releases. ## Related Units - [Docker Compose Reference](/software/devops/docker-compose-reference/2026)