Kafka Producer重连间隔能否自定义及Broker认证相关疑问

Great questions—let’s break them down clearly:

1. Can we customize the reconnection interval? #

Absolutely! Kafka provides several producer configuration parameters to control connection behavior and reconnection timing:

• connections.max.idle.ms: This sets the maximum time an idle connection can stay open before being closed. Once closed, the producer will re-establish a connection when it needs to send data again. The default value is 540000 ms (9 minutes). Increasing this value directly reduces how often reconnections happen.
• reconnect.backoff.ms: The initial delay before attempting to reconnect to a broker after a connection failure. Default is 50 ms.
• reconnect.backoff.max.ms: The maximum delay between reconnection attempts (the backoff increases exponentially up to this cap). Default is 1000 ms.

Tuning these parameters—especially connections.max.idle.ms—lets you tailor the reconnection frequency to your needs, helping reduce total connection counts and balance partition data as you noted.

2. Does authentication occur on every (re)connection when brokers have authentication enabled? #

Yes, every new connection (including reconnections) requires a full authentication handshake with the broker. Kafka connections are stateless, so there’s no persistent session across connection closures.

If you’re using LDAP-backed authentication (like SASL PLAIN with LDAP), each reconnection will trigger an LDAP lookup/validation. This can add noticeable load to your LDAP server if reconnections are frequent.

As you suspected, adjusting the reconnection interval (specifically increasing connections.max.idle.ms to keep connections open longer) is a solid solution here—it cuts down on the number of authentication requests sent to your LDAP server, easing its load.

内容的提问来源于stack exchange，提问作者Youngrok Ko