Kafka Topic副本因子为3仅单个分区偏移非零原因咨询

你看到的现象是Kafka的正常表现，核心原因是Kafka的分区是独立的消息存储单元，每个分区的消息生命周期完全独立，和副本因子无关，具体解释如下：

1. 分区与副本的核心区别 #

你混淆了「分区」和「分区副本」的概念：

• 副本因子为3，指的是每个单独的分区拥有3个副本（1个Leader副本 + 2个Follower副本），这些副本存储的是同一个分区的相同消息，并非把消息分散到3个不同的分区。
• Kafka的消息是按分区策略（如键哈希、轮询、指定分区等）写入到单个分区的，你看到的7条消息全部被写入了第9个分区，因此只有该分区的高水位会更新，其他无消息的分区高水位保持初始值0。

2. 高水位（High Watermark）的逻辑 #

高水位代表的是某个分区中，已经被该分区所有ISR（同步副本）成功复制的消息的最大偏移量+1：

• 只有当消息被当前分区的所有同步副本确认接收后，这个分区的高水位才会提升。
• 其他分区没有任何消息写入，自然没有需要同步的内容，高水位维持在初始的0值。

附：你获取的Kafka元数据 #

Topics:
  Topic: [HIDDEN]  Err: None
     Partition: 0  Leader: 123806  Replicas: [123806, 115260, 7075]  ISR: [123806, 115260, 7075]  Err: None
       Low watermark: 0  High watermark: 0 (difference: 0)
     Partition: 1  Leader: 7075  Replicas: [7075, 123806, 123244]  ISR: [7075, 123806, 123244]  Err: None
       Low watermark: 0  High watermark: 0 (difference: 0)
     Partition: 2  Leader: 123244  Replicas: [123244, 7075, 115250]  ISR: [123244, 7075, 115250]  Err: None
       Low watermark: 0  High watermark: 0 (difference: 0)
     Partition: 3  Leader: 115250  Replicas: [115250, 123244, 123790]  ISR: [115250, 123244, 123790]  Err: None
       Low watermark: 0  High watermark: 0 (difference: 0)
     Partition: 4  Leader: 123790  Replicas: [123790, 115250, 115260]  ISR: [123790, 115250, 115260]  Err: None
       Low watermark: 0  High watermark: 0 (difference: 0)
     Partition: 5  Leader: 115260  Replicas: [115260, 123790, 123806]  ISR: [115260, 123790, 123806]  Err: None
       Low watermark: 0  High watermark: 0 (difference: 0)
     Partition: 6  Leader: 123806  Replicas: [123806, 7075, 123244]  ISR: [123806, 7075, 123244]  Err: None
       Low watermark: 0  High watermark: 0 (difference: 0)
     Partition: 7  Leader: 7075  Replicas: [7075, 123244, 115250]  ISR: [7075, 123244, 115250]  Err: None
       Low watermark: 0  High watermark: 0 (difference: 0)
     Partition: 8  Leader: 123244  Replicas: [123244, 115250, 123790]  ISR: [123244, 115250, 123790]  Err: None
       Low watermark: 0  High watermark: 0 (difference: 0)
     Partition: 9  Leader: 115250  Replicas: [115250, 123790, 115260]  ISR: [115250, 123790, 115260]  Err: None
       Low watermark: 0  High watermark: 7 (difference: 7)
     Partition: 10  Leader: 123790  Replicas: [123790, 115260, 123806]  ISR: [123790, 115260, 123806]  Err: None
       Low watermark: 0  High watermark: 0 (difference: 0)
     Partition: 11  Leader: 115260  Replicas: [115260, 123806, 7075]  ISR: [115260, 123806, 7075]  Err: None
       Low watermark: 0  High watermark: 0 (difference: 0)
     Partition: 12  Leader: 123806  Replicas: [123806, 115250, 123790]  ISR: [123806, 115250, 123790]  Err: None
       Low watermark: 0  High watermark: 0 (difference: 0)
     Partition: 13  Leader: 7075  Replicas: [7075, 123790, 115260]  ISR: [7075, 123790, 115260]  Err: None
       Low watermark: 0  High watermark: 0 (difference: 0)
     Total message count: 7

内容的提问来源于stack exchange，提问作者user2138149