Docker中无外部卷挂载的Kafka仍保留旧主题问题排查

问题描述 #

通过Docker Compose运行Kafka，仅挂载/var/run/docker.sock，未配置其他外部卷。关闭栈后销毁容器并修剪容器卷，重启时却发现Kafka会重建之前的旧主题（原4分区主题变为1分区），同时__consumer_offsets也会被重建。已设置KAFKA_CFG_AUTO_CREATE_TOPICS_ENABLE: false但无效果，无法定位容器外的持久存储来源。

相关日志与配置 #

Kafka启动日志 #

...
2024-05-17 08:57:37 [2024-05-17 12:57:37,021] INFO [KafkaServer id=1001] started (kafka.server.KafkaServer)
2024-05-17 08:57:37 [2024-05-17 12:57:37,095] INFO [broker-1001-to-controller-send-thread]: Recorded new controller, from now on will use broker kafka:9093 (id: 1001 rack: null) (kafka.server.BrokerToControllerRequestThread)
2024-05-17 08:57:37 [2024-05-17 12:57:37,234] INFO Creating topic my-topic with configuration {} and initial partition assignment HashMap(0 -> ArrayBuffer(1001)) (kafka.zk.AdminZkClient)
2024-05-17 08:57:37 [2024-05-17 12:57:37,274] INFO Creating topic __consumer_offsets with configuration {compression.type=producer, cleanup.policy=compact, segment.bytes=104857600} and initial partition assignment HashMap(0 -> ArrayBuffer(1001), 1 -> ArrayBuffer(1001), 2 -> ArrayBuffer(1001), 3 -> ArrayBuffer(1001), 4 -> ArrayBuffer(1001), 5 -> ArrayBuffer(1001), 6 -> ArrayBuffer(1001), 7 -> ArrayBuffer(1001), 8 -> ArrayBuffer(1001), 9 -> ArrayBuffer(1001), 10 -> ArrayBuffer(1001), 11 -> ArrayBuffer(1001), 12 -> ArrayBuffer(1001), 13 -> ArrayBuffer(1001), 14 -> ArrayBuffer(1001), 15 -> ArrayBuffer(1001), 16 -> ArrayBuffer(1001), 17 -> ArrayBuffer(1001), 18 -> ArrayBuffer(1001), 19 -> ArrayBuffer(1001), 20 -> ArrayBuffer(1001), 21 -> ArrayBuffer(1001), 22 -> ArrayBuffer(1001), 23 -> ArrayBuffer(1001), 24 -> ArrayBuffer(1001), 25 -> ArrayBuffer(1001), 26 -> ArrayBuffer(1001), 27 -> ArrayBuffer(1001), 28 -> ArrayBuffer(1001), 29 -> ArrayBuffer(1001), 30 -> ArrayBuffer(1001), 31 -> ArrayBuffer(1001), 32 -> ArrayBuffer(1001), 33 -> ArrayBuffer(1001), 34 -> ArrayBuffer(1001), 35 -> ArrayBuffer(1001), 36 -> ArrayBuffer(1001), 37 -> ArrayBuffer(1001), 38 -> ArrayBuffer(1001), 39 -> ArrayBuffer(1001), 40 -> ArrayBuffer(1001), 41 -> ArrayBuffer(1001), 42 -> ArrayBuffer(1001), 43 -> ArrayBuffer(1001), 44 -> ArrayBuffer(1001), 45 -> ArrayBuffer(1001), 46 -> ArrayBuffer(1001), 47 -> ArrayBuffer(1001), 48 -> ArrayBuffer(1001), 49 -> ArrayBuffer(1001)) (kafka.zk.AdminZkClient)
2024-05-17 08:57:37 [2024-05-17 12:57:37,303] INFO [ReplicaFetcherManager on broker 1001] Removed fetcher for partitions Set(my-topic-0) (kafka.server.ReplicaFetcherManager)
2024-05-17 08:57:37 [2024-05-17 12:57:37,353] INFO [Log partition=my-topic-0, dir=/kafka/kafka-logs-5d423ab1a90a] Loading producer state till offset 0 with message format version 2 (kafka.log.Log)
2024-05-17 08:57:37 [2024-05-17 12:57:37,357] INFO Created log for partition my-topic-0 in /kafka/kafka-logs-5d423ab1a90a/my-topic-0 with properties {} (kafka.log.LogManager)
2024-05-17 08:57:37 [2024-05-17 12:57:37,358] INFO [Partition my-topic-0 broker=1001] No checkpointed highwatermark is found for partition my-topic-0 (kafka.cluster.Partition)
2024-05-17 08:57:37 [2024-05-17 12:57:37,358] INFO [Partition my-topic-0 broker=1001] Log loaded for partition my-topic-0 with initial high watermark 0 (kafka.cluster.Partition)  
...

ZooKeeper启动日志 #

...
2024-05-17 08:57:37 2024-05-17 12:57:37,236 [myid:] - INFO  [ProcessThread(sid:0 cport:2181)::PrepRequestProcessor@653] - Got user-level KeeperException when processing sessionid:0x100095707690000 type:setData cxid:0x45 zxid:0x20 txntype:-1 reqpath:n/a Error Path:/config/topics/my-topic Error:KeeperErrorCode = NoNode for /config/topics/my-topic
2024-05-17 08:57:37 2024-05-17 12:57:37,275 [myid:] - INFO  [ProcessThread(sid:0 cport:2181)::PrepRequestProcessor@653] - Got user-level KeeperException when processing sessionid:0x100095707690000 type:setData cxid:0x4f zxid:0x26 txntype:-1 reqpath:n/a Error Path:/config/topics/__consumer_offsets Error:KeeperErrorCode = NoNode for /config/topics/__consumer_offsets
...

Docker Compose配置 #

version: "3.8"

services:
  zookeeper:
    image: wurstmeister/zookeeper:latest
    ports:
      - "2181:2181"

  kafka:
    image: wurstmeister/kafka:latest
    ports:
      - "9092:9092"
    expose:
      - "9093"
    environment:
      KAFKA_ADVERTISED_LISTENERS: INSIDE://kafka:9093,OUTSIDE://localhost:9092
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: INSIDE:PLAINTEXT,OUTSIDE:PLAINTEXT
      KAFKA_LISTENERS: INSIDE://0.0.0.0:9093,OUTSIDE://0.0.0.0:9092
      KAFKA_INTER_BROKER_LISTENER_NAME: INSIDE
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_CFG_DELETE_TOPIC_ENABLE: true
      KAFKA_CFG_AUTO_CREATE_TOPICS_ENABLE: false

    volumes:
      - /var/run/docker.sock:/var/run/docker.sock

问题分析 #

1. 环境变量配置错误：wurstmeister/kafka镜像使用的环境变量前缀是KAFKA_而非KAFKA_CFG_，你设置的KAFKA_CFG_AUTO_CREATE_TOPICS_ENABLE实际未生效，Kafka仍使用默认的自动创建主题逻辑。
2. ZooKeeper匿名卷未清理：虽然你修剪了容器卷，但ZooKeeper容器默认会创建匿名卷存储数据，主题元信息实际保存在ZooKeeper中，重启时Kafka从ZooKeeper读取旧主题的元数据（但可能因为ZooKeeper数据不完整，导致分区数变为1）。
3. __consumer_offsets重建逻辑：当Kafka检测到ZooKeeper中没有__consumer_offsets主题时，会自动创建该主题，这是Kafka的内置机制，不受auto.create.topics.enable参数控制。

解决办法 #

1. 修正Kafka环境变量 #

将KAFKA_CFG_前缀改为KAFKA_，确保配置生效：

environment:
  # ...其他配置不变
  KAFKA_DELETE_TOPIC_ENABLE: true
  KAFKA_AUTO_CREATE_TOPICS_ENABLE: false

2. 彻底清理ZooKeeper数据 #

停止并删除所有相关容器后，清理ZooKeeper的匿名卷：

# 停止并删除容器
docker-compose down -v
# 清理所有未使用的卷（可选，确保无残留）
docker volume prune -f

3. 显式配置ZooKeeper卷（可选） #

如果需要更明确的控制，可给ZooKeeper配置命名卷，方便后续清理：

services:
  zookeeper:
    image: wurstmeister/zookeeper:latest
    ports:
      - "2181:2181"
    volumes:
      - zookeeper-data:/data

volumes:
  zookeeper-data:

清理时只需删除该命名卷：docker volume rm zookeeper-data

4. 检查外部依赖 #

确认没有其他外部ZooKeeper实例被当前Kafka连接，确保Kafka仅使用Compose中定义的ZooKeeper服务。

内容的提问来源于stack exchange，提问作者etcario