Apache Kafka主题消息记录被截断问题求助：配置排查指引

Hey there, let's figure out why your Kafka messages are getting truncated and how to fix this—this is a super common gotcha, so I’ll walk you through the key configs to check on both the broker and producer side, plus how to troubleshoot step by step.

You can’t just tweak one side and call it done—both the Kafka cluster and your producer need to be configured to handle larger messages. Here’s what to look for:

Broker-Side Configs #

These settings control what the Kafka cluster will accept and replicate:

• message.max.bytes: This is the maximum size (in bytes) of a single message that the broker will accept. The default is usually ~1MB (1048588 bytes). If your message exceeds this, the broker will either truncate it or reject it entirely. Update this to a value larger than your message size (e.g., 5MB = 5242880 bytes).
• replica.fetch.max.bytes: This sets the maximum size of a message that replicas can fetch from the leader broker. It must be at least as large as message.max.bytes—if it’s smaller, replica sync will fail, and you might end up with incomplete or missing messages across your cluster.
• Topic-level override: Don’t forget to check if your specific topic has its own max.message.bytes setting (this overrides the broker’s global config). Run this command to verify:

  kafka-topics.sh --describe --topic your-topic-name --bootstrap-server your-broker:9092
  

  Look for the max.message.bytes entry in the output—if it’s smaller than your message size, update it with:

  kafka-configs.sh --alter --topic your-topic-name --add-config max.message.bytes=5242880 --bootstrap-server your-broker:9092
  

Producer-Side Configs #

Your producer needs to be allowed to send large messages in the first place:

• max.request.size: This is the maximum size (in bytes) of a single request the producer will send to the broker. It must be equal to or larger than your message size (match the message.max.bytes value you set on the broker). The default is also ~1MB, so this is a common culprit if you haven’t adjusted it.
• buffer.memory: While not directly responsible for truncation, this sets the total memory the producer uses to buffer messages. Make sure it’s large enough to hold your big message without hitting memory limits (e.g., if you’re sending 5MB messages, set this to at least 64MB or higher depending on your throughput).

1. Confirm your actual message size: First, check exactly how big your message is. For example, in Java you can use yourMessageString.getBytes().length, or in Python len(your_message.encode('utf-8')). This tells you the minimum value your configs need to support.
2. Check broker logs: Look for error messages like "Message size too large" in the broker logs (usually in kafka/logs/server.log). This will tell you if the broker is rejecting or truncating the message because it exceeds message.max.bytes.
3. Check producer logs: If the producer is truncating the message before sending, you might see warnings or errors about exceeding max.request.size.
4. Test with a known large message: Send a test message of a fixed size (e.g., 5MB of random data) and use the console consumer to verify it arrives intact:

   # Send test message
   printf '%*s' 5242880 '' | kafka-console-producer.sh --broker-list your-broker:9092 --topic your-topic-name
   # Consume and check size
   kafka-console-consumer.sh --bootstrap-server your-broker:9092 --topic your-topic-name --from-beginning --max-messages 1 | wc -c
   

   If the output matches 5242880, your configs are working.

• Only updating one side: If you set message.max.bytes on the broker but leave max.request.size at 1MB on the producer, the producer will still truncate the message before sending.
• Mismatched replica config: Forgetting to update replica.fetch.max.bytes can lead to messages not replicating properly, so even if the leader has the full message, replicas might have truncated versions.
• Unit confusion: Don’t mix up KB and MB—1MB is 1024*1024 = 1048576 bytes, not 1024. Double-check your config values to make sure you’re using the right units.

内容的提问来源于stack exchange，提问作者Raj Ram