升级Spark与Job Server后创建metastore_db数据库失败求助

Hey, I’ve dealt with similar headaches when upgrading Job Server alongside newer Spark versions, so let’s break down what’s causing this ERROR XBM0H and how to fix it. That error code is actually from Apache Derby—even though you configured MySQL as your JobSqlDao store, there’s probably a conflict where Job Server is still trying to initialize Derby under the hood, or the SessionContextFactory has compatibility quirks with Spark 2.3.0.

Here are the step-by-step fixes to try:

1. Double-Check Your MySQL JobSqlDao Configuration #

Make sure your local.conf fully overrides the default Derby setup for JobSqlDao—partial configs often lead to fallback behavior. Here’s a complete example to adapt:

jobserver {
  dao = spark.jobserver.io.JobSqlDao
  sql {
    driver = com.mysql.cj.jdbc.Driver  # Use the CJ driver for Spark 2.3+; older com.mysql.jdbc.Driver works too
    url = "jdbc:mysql://YOUR_MYSQL_HOST:3306/jobserver?useSSL=false&serverTimezone=UTC"
    user = YOUR_DB_USERNAME
    password = YOUR_DB_PASSWORD
    dbcp {
      maxActive = 20
      maxIdle = 10
      minIdle = 2
      initialSize = 3
    }
  }
}

sql-context {
  context-factory = spark.jobserver.context.SessionContextFactory
}

Pro tip: Pair this with mysql-connector-java-8.0.11 or newer—this version plays nicely with Spark 2.3.0’s dependencies.

2. Test with SqlContextFactory to Rule Out Context Compatibility #

The SessionContextFactory in the 0.8.1-SNAPSHOT build might have unaddressed compatibility issues with Spark 2.3.0. Swap it out temporarily to verify:

sql-context {
  context-factory = spark.jobserver.context.SqlContextFactory
}

If the request works after this change, the problem is definitely with the SessionContextFactory. You can either:

• Pull the latest master branch of Job Server and recompile (there might be fixes for Spark 2.3)
• Stick with SqlContextFactory temporarily, or build a custom SessionContextFactory that’s compatible with Spark 2.3’s session handling

3. Clean Up Temporary Files and MySQL Initialization Data #

Sometimes Job Server leaves behind local Derby files even when you configure MySQL, causing initialization conflicts. Do this cleanup:

• Stop the Job Server service
• Delete the derby.log and metastore_db folder in your Job Server working directory
• Drop and recreate the jobserver database in MySQL (or truncate all tables) to let Job Server reinitialize the schema
• Restart the service and try your request again

4. Resolve Dependency Conflicts Between Spark and Job Server #

Spark 2.3.0 includes its own Derby and JDBC dependencies that might clash with Job Server’s. Add an exclusion for Derby when starting Job Server:

spark-submit --packages spark.jobserver:job-server_2.11:0.8.1-SNAPSHOT \
  --exclude-org org.apache.derby \
  --class spark.jobserver.JobServer \
  your-job-server-jar-file.jar

If you’re building Job Server from source, you can also add the exclusion in build.sbt before compiling.

内容的提问来源于stack exchange，提问作者user1933178