Java 8与Spark 3.2.0兼容的com.datastax.oss版本及依赖问题咨询

API 探路者

2026-7-22

问题分析与解决方案

问题背景

需确认Java 8、Spark 3.2.0兼容的com.datastax.oss版本
不明确Spark与Spring Boot的兼容关系
通过Spark查询Cassandra时抛出NoClassDefFoundError错误

错误堆栈信息

java.lang.NoClassDefFoundError: com/datastax/oss/driver/internal/core/util/collection/SimpleQueryPlan
    at com.datastax.spark.connector.cql.LocalNodeFirstLoadBalancingPolicy.newQueryPlan(LocalNodeFirstLoadBalancingPolicy.scala:122) ~[spark-cassandra-connector-driver_2.12-3.2.0.jar:3.2.0]
    at com.datastax.oss.driver.internal.core.metadata.LoadBalancingPolicyWrapper.newQueryPlan(LoadBalancingPolicyWrapper.java:155) ~[java-driver-core-4.9.0.jar:na]
    at com.datastax.oss.driver.internal.core.cql.CqlRequestHandler.onThrottleReady(CqlRequestHandler.java:211) ~[java-driver-core-4.9.0.jar:na]
    at com.datastax.oss.driver.internal.core.session.throttling.PassThroughRequestThrottler.register(PassThroughRequestThrottler.java:52) ~[java-driver-core-4.9.0.jar:na]

当前Maven POM配置

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.4.0</version>
        <relativePath /> <!-- lookup parent from repository -->
    </parent>
    <groupId>com.valoores.cassandra</groupId>
    <artifactId>cassandra-spark</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <name>cassandra-spark</name>
    <description>datacrowd-cassandra</description>
    <properties>
        <java.version>8</java.version>
    </properties>
    <dependencies>

        <dependency>
            <groupId>org.hibernate</groupId>
            <artifactId>hibernate-core</artifactId>
            <version>5.4.0.Final</version>
        </dependency>

        <dependency>
            <groupId>org.json</groupId>
            <artifactId>json</artifactId>
            <version>20210307</version>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-devtools</artifactId>
            <scope>runtime</scope>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-security</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.security</groupId>
            <artifactId>spring-security-test</artifactId>
            <scope>test</scope>
        </dependency>

        <dependency>
            <groupId>org.springdoc</groupId>
            <artifactId>springdoc-openapi-ui</artifactId>
            <version>1.6.0</version>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-cassandra</artifactId>
        </dependency>

        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-core_2.12</artifactId>
            <version>3.2.0</version>
        </dependency>
        <dependency>
            <groupId>com.datastax.spark</groupId>
            <artifactId>spark-cassandra-connector_2.12</artifactId>
            <version>3.2.0</version>
        </dependency>
        <dependency>
            <groupId>com.datastax.spark</groupId>
            <artifactId>spark-cassandra-connector-driver_2.12</artifactId>
            <version>3.2.0</version>
        </dependency>
        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-sql_2.12</artifactId>
            <version>3.2.0</version>
        </dependency>
 
    <dependency>
        <groupId>com.datastax.oss</groupId>
        <artifactId>java-driver-core</artifactId>
        <version>4.9.0</version>
    </dependency>
    <dependency>
        <groupId>com.datastax.oss</groupId>
        <artifactId>java-driver-core-shaded</artifactId>
        <version>4.9.0</version>
    </dependency>
    <dependency>
        <groupId>com.datastax.oss</groupId>
        <artifactId>java-driver-query-builder</artifactId>
        <version>4.9.0</version>
    </dependency>
    <dependency>
        <groupId>com.datastax.oss</groupId>
        <artifactId>java-driver-mapper-runtime</artifactId>
        <version>4.9.0</version>
    </dependency>  
    <dependency>
    <groupId>com.datastax.oss</groupId>
    <artifactId>java-driver-shaded-guava</artifactId>
    <version>25.1-jre</version>
</dependency>


    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
                <configuration>
                    <excludes>
                        <exclude>
                            <groupId>org.projectlombok</groupId>
                            <artifactId>lombok</artifactId>
                        </exclude>
                    </excludes>
                </configuration>
            </plugin>
        </plugins>


    </build>

</project>

问题根源

驱动版本不匹配：spark-cassandra-connector 3.2.0官方依赖的DataStax Java Driver版本为4.13.0，而你使用的4.9.0中不存在SimpleQueryPlan类，该类是在4.10+版本中新增的，导致类找不到错误。
依赖冲突：同时引入java-driver-core和java-driver-core-shaded会引发类加载冲突；另外spring-boot-starter-data-cassandra自带的DataStax驱动版本与手动引入的版本不一致，也会导致冲突。
Spring Boot与Spark兼容性：Spring Boot 2.4.x和Spark 3.2.0是兼容的，两者均支持Java 8。

修复方案

1. 统一DataStax驱动版本

将所有com.datastax.oss依赖的版本改为4.13.0，同时：

移除java-driver-core，仅保留java-driver-core-shaded（避免与Spark依赖的第三方库冲突）
移除java-driver-shaded-guava，因为shaded驱动已包含适配版本的Guava

2. 解决Spring Data Cassandra依赖冲突

通过Maven的dependencyManagement强制统一DataStax驱动版本，或者排除spring-boot-starter-data-cassandra自带的驱动依赖：

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-data-cassandra</artifactId>
    <exclusions>
        <exclusion>
            <groupId>com.datastax.oss</groupId>
            <artifactId>java-driver-core</artifactId>
        </exclusion>
        <exclusion>
            <groupId>com.datastax.oss</groupId>
            <artifactId>java-driver-query-builder</artifactId>
        </exclusion>
    </exclusions>
</dependency>

3. 修正后的核心依赖示例

<!-- Spark相关依赖 -->
<dependency>
    <groupId>org.apache.spark</groupId>
    <artifactId>spark-core_2.12</artifactId>
    <version>3.2.0</version>
</dependency>
<dependency>
    <groupId>com.datastax.spark</groupId>
    <artifactId>spark-cassandra-connector_2.12</artifactId>
    <version>3.2.0</version>
</dependency>
<dependency>
    <groupId>com.datastax.spark</groupId>
    <artifactId>spark-cassandra-connector-driver_2.12</artifactId>
    <version>3.2.0</version>
</dependency>
<dependency>
    <groupId>org.apache.spark</groupId>
    <artifactId>spark-sql_2.12</artifactId>
    <version>3.2.0</version>
</dependency>

<!-- DataStax驱动，统一为4.13.0 -->
<dependency>
    <groupId>com.datastax.oss</groupId>
    <artifactId>java-driver-core-shaded</artifactId>
    <version>4.13.0</version>
</dependency>
<dependency>
    <groupId>com.datastax.oss</groupId>
    <artifactId>java-driver-query-builder</artifactId>
    <version>4.13.0</version>
</dependency>
<dependency>
    <groupId>com.datastax.oss</groupId>
    <artifactId>java-driver-mapper-runtime</artifactId>
    <version>4.13.0</version>
</dependency>

<!-- Spring Data Cassandra，排除自带驱动 -->
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-data-cassandra</artifactId>
    <exclusions>
        <exclusion>
            <groupId>com.datastax.oss</groupId>
            <artifactId>java-driver-core</artifactId>
        </exclusion>
        <exclusion>
            <groupId>com.datastax.oss</groupId>
            <artifactId>java-driver-query-builder</artifactId>
        </exclusion>
    </exclusions>
</dependency>

版本兼容说明

Java 8 + Spark 3.2.0 对应的DataStax Java Driver版本为4.13.0（spark-cassandra-connector 3.2.0官方指定版本）
Spring Boot 2.4.x 与 Spark 3.2.0 兼容，两者均基于Java 8构建，无核心冲突

内容的提问来源于stack exchange，提问作者Nathalie

Java 8与Spark 3.2.0兼容的com.datastax.oss版本及依赖问题咨询

问题背景 #

错误堆栈信息 #

当前Maven POM配置 #

问题根源 #

修复方案 #

1. 统一DataStax驱动版本 #

2. 解决Spring Data Cassandra依赖冲突 #

3. 修正后的核心依赖示例 #

版本兼容说明 #

问题背景

错误堆栈信息

当前Maven POM配置

问题根源

修复方案

1. 统一DataStax驱动版本

2. 解决Spring Data Cassandra依赖冲突

3. 修正后的核心依赖示例

版本兼容说明