如何在Kafka 10.0.2节点上检查损坏的日志文件？

Absolutely, with Kafka 10.0.2 (part of the 0.10.0.x release line), you have reliable CLI and manual methods to verify if your log segments (the .log, .index, and .timeindex files in /var/kafka/kafka-logs/) are corrupted or missing critical data. Since you’ve already spotted index/timeindex corruption warnings in server.log, let’s break down how to validate these files properly:

Kafka ships with a dedicated utility via kafka-run-class.sh that’s designed to inspect and validate log segments. This is the most authoritative method for your use case.

Validate .log Files #

To check if a log file is intact and can be parsed correctly, run:

# First navigate to your Kafka bin directory
cd /path/to/kafka/bin

# Replace the file path with your target log segment
./kafka-run-class.sh kafka.tools.DumpLogSegments --files /var/kafka/kafka-logs/kuapi.jurg.pri.decoded-8/00000000000000000000.log --print-data-log

• If the file is healthy, it will print the raw records stored in the log.
• If corrupted, you’ll see explicit errors like CorruptRecordException, InvalidOffsetException, or messages about unreadable segment data.

Validate .index and .timeindex Files #

To check index file integrity, use the --dump-index flag to inspect the index entries:

./kafka-run-class.sh kafka.tools.DumpLogSegments --files /var/kafka/kafka-logs/kuapi.jurg.pri.decoded-8/00000000000000000000.index --dump-index

• Healthy index files will output a clean list of offset-position (for .index) or timestamp-offset (for .timeindex) pairs.
• Corrupted indexes will throw errors about invalid entries, mismatched offsets, or unreadable file headers.

Bulk Check All Files #

To avoid checking each file manually, use a simple shell script to scan every segment in your log directory:

#!/bin/bash
LOG_DIR="/var/kafka/kafka-logs"
KAFKA_BIN="/path/to/kafka/bin"

echo "Starting bulk validation of Kafka log segments..."
echo "-----------------------------------------------"

for topic_dir in "$LOG_DIR"/*/; do
    echo "Checking topic partition: $(basename "$topic_dir")"
    for segment_file in "$topic_dir"/*.log "$topic_dir"/*.index "$topic_dir"/*.timeindex; do
        # Skip if file doesn't exist (e.g., latest log might not have flushed indexes yet)
        [ -f "$segment_file" ] || continue
        
        echo "  Validating: $(basename "$segment_file")"
        "$KAFKA_BIN/kafka-run-class.sh" kafka.tools.DumpLogSegments --files "$segment_file" > /dev/null 2>&1
        
        if [ $? -ne 0 ]; then
            echo "  ❌ CORRUPTED: $(basename "$segment_file")"
        else
            echo "  ✅ OK: $(basename "$segment_file")"
        fi
    done
    echo "-----------------------------------------------"
done

Save this as validate-kafka-logs.sh, make it executable (chmod +x validate-kafka-logs.sh), and run it—it will flag any corrupted files automatically.

For quick preliminary checks without tools:

• Check file pairing: Every .log file should have matching .index and .timeindex files (except possibly the very latest log segment, which might not have flushed indexes yet). Missing paired files are a clear red flag.
• Verify index file sizes: In Kafka 10.0.2, .index and .timeindex files follow a fixed format:
  • Each index entry is 16 bytes (8 bytes for offset/timestamp + 8 bytes for position/offset)
  • Files start with a 4-byte version header
  • Valid index files should have a size of 4 + 16 * N (where N is the number of entries). Use stat -c %s <index-file> to check if the size fits this pattern.
• Cross-reference server.log: Use your existing corruption warnings to prioritize segments:

  grep -i "corrupt\|invalid\|index" /var/log/kafka/server.log | grep -E "([0-9]+\.(log|index|timeindex))"
  

This will pull out specific file paths mentioned in error messages, so you can focus on those first.

• Stick to the same version of Kafka tools: Don’t use newer Kafka binaries to validate 0.10.0.2 log segments—version mismatches can cause false positives or incorrect parsing.
• If you find corrupted files:
  • For partitions with multiple replicas: Delete the corrupted files and restart Kafka—healthy replicas will sync the missing data automatically.
  • For single-replica partitions: You’ll likely lose some data. Backup the corrupted files first, then delete them. Kafka will restart the partition from the last valid offset.

内容的提问来源于stack exchange，提问作者enodmilvado