Kafka升级疑问：升级后为何需设置inter.broker.protocol.version为新版本？

Great question! Let's break this down clearly to understand why explicitly setting the protocol version is recommended instead of just removing the parameter.

• Upgrade vs. Fresh Install Defaults
  For a fresh Kafka 2.3.0 installation, the default inter.broker.protocol.version is indeed 2.3. But when upgrading from 1.1.0, removing the parameter doesn't guarantee all brokers will immediately switch to the new protocol reliably. During the first rolling restart (with inter.broker.protocol.version=1.1), brokers initialized their communication context using the old protocol. Removing the parameter without explicitly setting it to 2.3 could lead to temporary inconsistencies—some brokers might retain cached old protocol settings, causing mixed protocol usage across the cluster.

• Activate 2.3-Specific Broker Features
  This setting isn't just about broker-to-broker communication; it unlocks new features introduced in Kafka 2.3.0. When you explicitly set inter.broker.protocol.version=2.3 and restart all brokers, Kafka runs internal checks to confirm all nodes are on the new protocol, then enables enhancements like optimized metadata propagation, improved request handling, and other version-specific optimizations. Removing the parameter might skip this activation step, leaving you without access to these improvements.

• Eliminate Rollback Ambiguity
  Explicitly setting the version to 2.3 acts as a clear marker that your cluster is fully upgraded and no longer needs to support 1.1.0 brokers. If you ever need to roll back (even if unlikely here), having this explicit setting makes it easy to verify the cluster's protocol state. Removing the parameter would leave you uncertain whether brokers are using the default 2.3 protocol or falling back to a cached old version.

• Guarantee Cluster Consistency
  Kafka's official upgrade steps prioritize explicit configuration to ensure uniform protocol usage across all nodes. Even if removing the parameter might work in some scenarios, explicit settings eliminate room for error. In a distributed system like Kafka, consistent protocol usage is critical for stability, performance, and avoiding hidden edge cases.

In short: while 2.3.0 brokers default to the 2.3 protocol for fresh installs, upgrading requires an explicit setting to ensure full, consistent adoption of the new protocol and its features. Skipping this step could lead to hidden inconsistencies or missed enhancements.

内容的提问来源于stack exchange，提问作者Yannick