如何通过Docker Compose部署KSQLDB与Kafka并解决连接异常

报错里的DUPLICATE_BROKER_REGISTRATION说明Kafka Broker尝试注册重复ID，KSQLDB连接失败是因为Kafka服务本身未正常启动。问题根源在于confluent-local镜像的默认配置冲突，且缺少必要的Kafka初始化参数。

以下是可正常运行Kafka + KSQLDB、支持流/表实验的完整配置：

version: "3.8"
services:
  zookeeper:
    image: confluentinc/cp-zookeeper:7.5.3
    container_name: zookeeper
    ports:
      - "2181:2181"
    environment:
      - ZOOKEEPER_CLIENT_PORT=2181
      - ZOOKEEPER_TICK_TIME=2000
    healthcheck:
      test: ["CMD", "zkServer.sh", "status"]
      interval: 10s
      timeout: 5s
      retries: 5

  kafka:
    image: confluentinc/cp-kafka:7.5.3
    container_name: kafka
    ports:
      - "9092:9092"
      - "29092:29092"
    environment:
      - KAFKA_BROKER_ID=1
      - KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181
      - KAFKA_LISTENERS=PLAINTEXT://0.0.0.0:9092,PLAINTEXT_HOST://0.0.0.0:29092
      - KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://kafka:9092,PLAINTEXT_HOST://localhost:29092
      - KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR=1
      - KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR=1
      - KAFKA_TRANSACTION_STATE_LOG_MIN_ISR=1
    depends_on:
      zookeeper:
        condition: service_healthy

  schema-registry:
    image: confluentinc/cp-schema-registry:7.5.3
    container_name: schema-registry
    ports:
      - "8081:8081"
    environment:
      - SCHEMA_REGISTRY_HOST_NAME=schema-registry
      - SCHEMA_REGISTRY_KAFKASTORE_BOOTSTRAP_SERVERS=kafka:9092
    depends_on:
      kafka:
        condition: service_started

  ksqldb-server:
    image: confluentinc/cp-ksqldb-server:7.5.3
    container_name: ksqldb-server
    ports:
      - "8088:8088"
    environment:
      - KSQL_BOOTSTRAP_SERVERS=kafka:9092
      - KSQL_KSQL_SERVICE_ID=ksqldb_demo_
      - KSQL_KSQL_LISTENERS=http://0.0.0.0:8088
      - KSQL_KSQL_SCHEMA_REGISTRY_URL=http://schema-registry:8081
      - KSQL_KSQL_LOGGING_PROCESSING_STREAM_AUTO_CREATE=true
      - KSQL_KSQL_LOGGING_PROCESSING_TOPIC_AUTO_CREATE=true
    depends_on:
      kafka:
        condition: service_started
      schema-registry:
        condition: service_started

• 替换confluent-local为标准cp-kafka镜像，避免默认配置冲突
• 添加ZooKeeper组件（Kafka 7.x版本仍依赖ZooKeeper，实验场景下更易配置）
• 明确Kafka监听地址，确保容器内部和外部客户端都能正常连接
• 为ZooKeeper添加健康检查，保证Kafka启动前依赖服务就绪
• 加入Schema Registry，满足KSQLDB流/表的序列化需求
• 开启KSQLDB自动创建处理流和主题的参数，简化实验流程

1. 将上述内容保存为docker-compose.yaml
2. 执行docker-compose up -d启动所有服务
3. 等待1-2分钟服务就绪后，通过docker exec -it ksqldb-server ksql http://localhost:8088进入KSQLDB CLI进行流/表实验

内容的提问来源于stack exchange，提问作者Nikolaj