Kafka持续报Node -1断开连接问题求助（Apple Silicon M1环境）

报错日志 #

Node -1 disconnected. 
14:30:07.192 trace-id: span-id: [kafka-producer-network-thread | telikos-activityplan-producer] WARN  o.apache.kafka.clients.NetworkClient - [Producer clientId=telikos-activityplan-producer] Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Broker may not be available. 
14:30:07.193 trace-id: span-id: [kafka-producer-network-thread | telikos-activityplan-producer] WARN  o.apache.kafka.clients.NetworkClient - [Producer clientId=telikos-activityplan-producer] Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected 

问题背景 #

Docker中ZooKeeper、Kafka Broker、Schema Registry均显示正常运行，但本地生产者连接localhost:9092时仅约1%概率成功，其余时间报上述连接断开错误，使用Apple Silicon M1设备，怀疑是否为设备特定问题。

Docker Compose配置 #

version: '2'
services:
  zookeeper:
    image: confluentinc/cp-zookeeper:7.0.1
    hostname: zookeeper
    container_name: zookeeperap2
    ports:
      - "2181:2181"
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_TICK_TIME: 2000

  broker:
    image: confluentinc/cp-server:7.0.1
    hostname: broker
    container_name: brokerap2
    depends_on:
      - zookeeper
    ports:
      - "9092:9092"
      - "9101:9101"
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: 'zookeeper:2181'
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://broker:29092,PLAINTEXT_HOST://localhost:9092
      KAFKA_METRIC_REPORTERS: io.confluent.metrics.reporter.ConfluentMetricsReporter
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 0
      KAFKA_CONFLUENT_LICENSE_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_CONFLUENT_BALANCER_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1
      KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1
      KAFKA_JMX_PORT: 9101
      KAFKA_JMX_HOSTNAME: localhost
      KAFKA_CONFLUENT_SCHEMA_REGISTRY_URL: http://schema-registry:8081
      CONFLUENT_METRICS_REPORTER_BOOTSTRAP_SERVERS: broker:29092
      CONFLUENT_METRICS_REPORTER_TOPIC_REPLICAS: 1
      CONFLUENT_METRICS_ENABLE: 'true'
      CONFLUENT_SUPPORT_CUSTOMER_ID: 'anonymous'


  schema-registry:
    image: confluentinc/cp-schema-registry:7.0.1
    hostname: schema-registry
    container_name: schema-registryap2
    depends_on:
      - broker
    ports:
      - "8081:8081"
    environment:
      SCHEMA_REGISTRY_HOST_NAME: schema-registry
      SCHEMA_REGISTRY_KAFKASTORE_BOOTSTRAP_SERVERS: 'broker:29092'
      SCHEMA_REGISTRY_LISTENERS: http://0.0.0.0:8081

  control-center:
    image: confluentinc/cp-enterprise-control-center:7.0.1
    hostname: control-center
    container_name: control-centerap2
    depends_on:
      - broker
      - schema-registry
    ports:
      - "9021:9021"
    environment:
      CONTROL_CENTER_BOOTSTRAP_SERVERS: 'broker:29092'
      CONTROL_CENTER_SCHEMA_REGISTRY_URL: "http://schema-registry:8081"
      CONTROL_CENTER_REPLICATION_FACTOR: 1
      CONTROL_CENTER_INTERNAL_TOPICS_PARTITIONS: 1
      CONTROL_CENTER_MONITORING_INTERCEPTOR_TOPIC_PARTITIONS: 1
      CONFLUENT_METRICS_TOPIC_REPLICATION: 1
      PORT: 9021

排查方案 #

• 镜像兼容性调整：当前使用的Confluent 7.0.1镜像仅支持x86架构，在M1上依赖Rosetta转译易出现网络不稳定。更换为7.3.0及以上版本的arm64兼容镜像（如confluentinc/cp-server:7.3.0），或在Docker Desktop设置中开启"Use Rosetta for x86/amd64 emulation on Apple Silicon"。
• 修正监听地址映射：修改broker的KAFKA_ADVERTISED_LISTENERS配置，将PLAINTEXT_HOST://localhost:9092改为PLAINTEXT_HOST://host.docker.internal:9092，M1 Docker中host.docker.internal比localhost更可靠地映射宿主机地址，修改后重启broker容器。
• 检查端口占用：执行lsof -i :9092确认宿主机9092端口是否被其他进程占用，若占用则修改Docker Compose中broker的端口映射（如9093:9092），同时同步更新KAFKA_ADVERTISED_LISTENERS中的端口。
• 调整生产者连接参数：在生产者配置中增大connection.timeout.ms至5000，reconnect.backoff.ms至1000，减少短暂网络波动引发的连接失败。
• 查看Broker端日志：执行docker logs brokerap2 | grep "Connection"查看broker的网络交互日志，定位是否存在主动拒绝连接或异常断开的情况。

内容的提问来源于stack exchange，提问作者MoonPie