Kafka Kraft集群节点频繁断开,多分区生产消费异常求助
Docker Compose部署Bitnami Kafka Kraft集群异常问题
环境背景
计划基于Bitnami Kafka 3.3.1版本搭建3节点Kraft集群,实际使用bitnami/kafka:3.4.0-debian-11-r11镜像部署。该集群在Kubernetes环境可正常运行,但Docker Compose部署时出现多类异常。
异常现象
- 节点频繁断开,Docker日志输出
INFO [RaftManager nodeId=2] Node 1 disconnected.,Python客户端无法正常生产或消费消息 - 执行
kafka-console-producer.sh时触发NOT_LEADER_OR_FOLLOWER错误 kafka-console-consumer.sh运行状态不稳定,时好时坏- 单分区主题可正常工作,但多分区主题会触发上述所有异常,Python客户端消费者无法监听消息
已尝试更换镜像版本、更换Python客户端,问题仍未解决。
Broker1的Docker Compose配置
broker1: image: bitnami/kafka:3.4.0-debian-11-r11 hostname: broker1 container_name: broker1 ports: - "9092:9092" - "9093:9093" environment: KAFKA_ENABLE_KRAFT: 1 ALLOW_PLAINTEXT_LISTENER: 1 KAFKA_KRAFT_CLUSTER_ID: ${CLUSTER_ID} KAFKA_NODE_ID: 1 KAFKA_BROKER_ID: 1 KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP: CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT KAFKA_CFG_ADVERTISED_LISTENERS: PLAINTEXT://broker1:29092,PLAINTEXT_HOST://${EXTERNAL_IP}:9092 KAFKA_CFG_CONTROLLER_QUORUM_VOTERS: 1@${EXTERNAL_IP}:9093,2@${EXTERNAL_IP}:9083,3@${EXTERNAL_IP}:9073,4@${EXTERNAL_IP}:9063 KAFKA_CFG_LISTENERS: PLAINTEXT://localhost:29092,CONTROLLER://:9093,PLAINTEXT_HOST://:9092 KAFKA_CFG_CONTROLLER_LISTENER_NAMES: CONTROLLER KAFKA_CFG_PROCESS_ROLES: broker,controller KAFKA_CFG_GROUP_INITIAL_REBALANCE_DELAY_MS: 0 KAFKA_LOG_DIRS: /bitnami/kafka/kafka-logs
内容的提问来源于stack exchange,提问作者jldragon
相关产品推荐
相关产品推荐

