Spark、Kafka与HBase(Phoenix)集成遇Outdated jars错误，求解决方案

Hey there! Let's fix this Phoenix outdated jars issue you're hitting with your Kafka+Spark+HBase setup. I've dealt with this exact problem before, so here's a step-by-step breakdown to get you back on track:

This ERROR 2007 (INT09): Outdated jars error happens entirely because of a version mismatch between the Phoenix client jars your Spark application is using, and the Phoenix jars deployed on your HBase cluster nodes. The versions need to be identical across all components—no exceptions.

• First: Verify Version Consistency
  Start by checking the Phoenix version your Spark project depends on (look in your pom.xml, build.gradle, or dependency management file). Then, log into each HBase cluster node and check the $HBASE_HOME/lib directory—make sure all Phoenix-related jars here match the version your Spark app is using. For example, if your client uses phoenix-5.0.0-HBase-2.0, every node's lib folder must have the same version, no old or mismatched jars allowed.

• Second: Update HBase Cluster's Phoenix Jars
  If the cluster's jars are outdated, copy the correct version of Phoenix core jars (like phoenix-core-<version>.jar and phoenix-server-<version>.jar) to $HBASE_HOME/lib on every HBase node. Then delete all old, mismatched Phoenix jars from those directories.

• Third: Restart the Entire HBase Cluster
  Jar changes won't take effect until you restart HBase. Run these commands on your cluster's master node:

  # Stop HBase services
  stop-hbase.sh
  # Start HBase services back up
  start-hbase.sh
  

  Wait for all HMaster and RegionServer processes to fully restart before moving on.

• Fourth: Clean Up Spark Dependencies
  Make sure your Spark project doesn't have conflicting HBase/Phoenix dependencies. When submitting your Spark job, explicitly include the matching Phoenix jars using the --jars parameter to avoid version conflicts with cluster-provided libraries. Example submit command:

  spark-submit \
    --class com.your.package.YourMainClass \
    --jars phoenix-core-5.0.0-HBase-2.0.jar,phoenix-server-5.0.0-HBase-2.0.jar \
    your-spark-application.jar
  

• You don't usually need to mess with ZooKeeper configs here, but a full HBase restart will refresh any Phoenix-related ZooKeeper nodes automatically.
• Since you're using the "thick" JDBC client (the jdbc:phoenix:... connection string), you don't have to worry about Thin client versions—just keep the cluster and Spark client jars in sync.

内容的提问来源于stack exchange，提问作者I Soft