Spring Boot3.0.4整合Apache Spark3.3.2启动失败：缺失javax/servlet/Servlet

问题背景 #

基于Java 17搭建Spring Boot 3.0.4与Apache Spark 3.3.2的整合项目，容器启动时抛出类未找到异常，尝试添加javax.servlet依赖后问题依旧。

完整异常日志 #

Error starting ApplicationContext. To display the condition evaluation report re-run your application with 'debug' enabled.
2023-03-09T14:15:02.446-06:00 ERROR 8072 --- [           main] o.s.boot.SpringApplication               : Application run failed

org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'javaSparkContext' defined in class path resource [com/engin/controller/SparkContextManager.class]: Failed to instantiate [org.apache.spark.api.java.JavaSparkContext]: Factory method 'javaSparkContext' threw exception with message: javax/servlet/Servlet
    at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:657) ~[spring-beans-6.0.6.jar:6.0.6]
    at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:645) ~[spring-beans-6.0.6.jar:6.0.6]
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1324) ~[spring-beans-6.0.6.jar:6.0.6]
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1161) ~[spring-beans-6.0.6.jar:6.0.6]
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:561) ~[spring-beans-6.0.6.jar:6.0.6]
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:521) ~[spring-beans-6.0.6.jar:6.0.6]
    at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:326) ~[spring-beans-6.0.6.jar:6.0.6]
    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234) ~[spring-beans-6.0.6.jar:6.0.6]
    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:324) ~[spring-beans-6.0.6.jar:6.0.6]
    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:200) ~[spring-beans-6.0.6.jar:6.0.6]
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:961) ~[spring-beans-6.0.6.jar:6.0.6]
    at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:917) ~[spring-context-6.0.6.jar:6.0.6]
    at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:584) ~[spring-context-6.0.6.jar:6.0.6]
    at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:146) ~[spring-boot-3.0.4.jar:3.0.4]
    at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:732) ~[spring-boot-3.0.4.jar:3.0.4]
    at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:434) ~[spring-boot-3.0.4.jar:3.0.4]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:310) ~[spring-boot-3.0.4.jar:3.0.4]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1304) ~[spring-boot-3.0.4.jar:3.0.4]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1293) ~[spring-boot-3.0.4.jar:3.0.4]
    at com.engin.EnginServiceV6Application.main(EnginServiceV6Application.java:12) ~[classes/:na]
Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.apache.spark.api.java.JavaSparkContext]: Factory method 'javaSparkContext' threw exception with message: javax/servlet/Servlet
    at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:171) ~[spring-beans-6.0.6.jar:6.0.6]
    at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:653) ~[spring-beans-6.0.6.jar:6.0.6]
    ... 19 common frames omitted
Caused by: java.lang.NoClassDefFoundError: javax/servlet/Servlet
    at org.apache.spark.ui.SparkUI$.create(SparkUI.scala:223) ~[spark-core_2.13-3.3.2.jar:3.3.2]
    at org.apache.spark.SparkContext.<init>(SparkContext.scala:484) ~[spark-core_2.13-3.3.2.jar:3.3.2]
    at org.apache.spark.api.java.JavaSparkContext.<init>(JavaSparkContext.scala:58) ~[spark-core_2.13-3.3.2.jar:3.3.2]
    at com.engin.controller.SparkContextManager.javaSparkContext(SparkContextManager.java:39) ~[classes/:na]
    at com.engin.controller.SparkContextManager$$SpringCGLIB$$0.CGLIB$javaSparkContext$1(<generated>) ~[classes/:na]
    at com.engin.controller.SparkContextManager$$SpringCGLIB$$2.invoke(<generated>) ~[classes/:na]
    at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:258) ~[spring-core-6.0.6.jar:6.0.6]
    at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:331) ~[spring-context-6.0.6.jar:6.0.6]
    at com.engin.controller.SparkContextManager$$SpringCGLIB$$0.javaSparkContext(<generated>) ~[classes/:na]
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) ~[na:na]
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
    at java.base/java.lang.reflect.Method.invoke(Method.java:568) ~[na:na]
    at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:139) ~[spring-beans-6.0.6.jar:6.0.6]
    ... 20 common frames omitted
Caused by: java.lang.ClassNotFoundException: javax.servlet.Servlet
    at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641) ~[na:na]
    at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188) ~[na:na]
    at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:520) ~[na:na]
    ... 34 common frames omitted

当前pom.xml核心依赖片段 #

<parent>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-parent</artifactId>
    <version>3.0.4</version>
</parent>
<dependencies>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-web</artifactId>
    </dependency>
    <dependency>
        <groupId>org.apache.spark</groupId>
        <artifactId>spark-core_${scala-tool.version}</artifactId>
        <version>${apache-spark.version}</version>
    </dependency>
    <dependency>
        <groupId>org.apache.spark</groupId>
        <artifactId>spark-sql_${scala-tool.version}</artifactId>
        <version>${apache-spark.version}</version>
    </dependency>
</dependencies>

问题原因 #

Spring Boot 3.x 遵循Jakarta EE 9规范，Servlet相关API的包名从javax.servlet变更为jakarta.servlet；而Apache Spark 3.3.2仍依赖旧版javax.servlet API，两者类路径不兼容，导致Spark初始化Web UI时找不到javax.servlet.Servlet类。

解决方法 #

方法1：禁用Spark UI（推荐，无依赖冲突） #

若业务不需要Spark的Web监控界面，在创建SparkContext时添加配置关闭UI，避免加载Servlet相关类：

@Configuration
public class SparkContextManager {
    @Bean
    public JavaSparkContext javaSparkContext() {
        SparkConf conf = new SparkConf()
                .setAppName("EnginServiceSpark")
                .setMaster("local[*]")
                .set("spark.ui.enabled", "false"); // 禁用Spark UI
        return new JavaSparkContext(conf);
    }
}

方法2：添加旧版javax.servlet依赖 #

如果需要保留Spark UI，在pom.xml中显式添加javax.servlet-api依赖，为Spark提供所需的旧版类：

<dependencies>
    <!-- 已存在的依赖 -->
    <dependency>
        <groupId>javax.servlet</groupId>
        <artifactId>javax.servlet-api</artifactId>
        <version>4.0.1</version>
        <scope>provided</scope>
    </dependency>
</dependencies>

注意：若provided scope导致类无法加载，可临时改为compile，但需注意与Spring Boot内置的Jakarta Servlet依赖可能存在的冲突。

方法3：升级Spark版本（长期方案） #

Apache Spark 3.4.0及以上版本已支持Jakarta EE规范，若项目允许升级，将Spark版本更新至3.4.0+，即可直接兼容Spring Boot 3.x，无需额外添加兼容依赖。

内容的提问来源于stack exchange，提问作者sam