系统重启后Kafka主题丢失，如何找回已创建的所有主题？

Hey there, let's figure out why your Kafka topics disappeared after restarting your system. Here's a step-by-step breakdown to troubleshoot and get them back:

Wait, your third command has an extra test parameter that shouldn't be there. The correct way to list topics using Zookeeper is:

bin/kafka-topics.sh --list --zookeeper localhost:2181

If you're running a newer Kafka version (2.2+), it's better to use the bootstrap server instead (Zookeeper-based commands are deprecated):

bin/kafka-topics.sh --list --bootstrap-server localhost:9092

Run the corrected command first—maybe the topics are there, you just didn't list them properly.

Kafka stores actual topic data in the directory specified by log.dirs in your config/server.properties file. The default is often /tmp/kafka-logs, which is a problem because many systems clear /tmp on restart.

• Open config/server.properties and look for the log.dirs line
• Navigate to that folder using cd and list the contents with ls
• If you see folders named after your topics (like my-topic-0 where my-topic is your topic name and 0 is the partition), the data is still there—Kafka just isn't recognizing it. If those folders are gone, the data was likely deleted when /tmp was cleared.

Kafka stores topic metadata (like names, partitions, replication factors) in Zookeeper. Let's check if that metadata exists:

• Launch the Zookeeper shell:

  bin/zookeeper-shell.sh localhost:2181
  

• Once connected, run this command to list all topics stored in Zookeeper:

  ls /brokers/topics
  

• If your topics show up here, the issue is with Kafka not reading them properly. If they don't, the metadata was lost—but we can recover if the log data still exists.

If you found the topic log folders on disk but Zookeeper has no record of the topics, you can recreate the topics with the exact same configuration, and Kafka will automatically attach to the existing data:

1. First, note the original partition count and replication factor (you can get this from the log folder names—e.g., test-0 means partition 0, so at least 1 partition; if you had multiple replicas, you'd see the folder on multiple brokers)
2. Recreate the topic using:

   bin/kafka-topics.sh --create --bootstrap-server localhost:9092 --topic YOUR_TOPIC_NAME --partitions X --replication-factor Y
   

   Replace YOUR_TOPIC_NAME, X, and Y with your topic's name, original partition count, and replication factor.
3. Run the topic list command again—your topic should be back, and all existing data should be accessible.

• Don't use /tmp for Kafka or Zookeeper storage: Update log.dirs in server.properties to a persistent directory like /var/lib/kafka/logs, and dataDir in zookeeper.properties to something like /var/lib/zookeeper/data. Temporary directories get cleared on restart, which wipes your data and metadata.
• Shut down services properly: Always stop Kafka first with bin/kafka-server-stop.sh, then Zookeeper with bin/zookeeper-server-stop.sh before rebooting your system. This ensures all data and metadata is saved correctly.

内容的提问来源于stack exchange，提问作者hasherBaba