SpringBoot Kafka基于OpenJDK17容器化时Snappy平台不兼容问题求助

基于Spring Kafka实现的消费者应用，使用OpenJDK17容器化部署时会100%触发以下错误，但使用其他JDK基础镜像或非容器化环境时无此问题。手动引入指定版本的snappy-java依赖也无法解决。

错误日志 #

ERROR 1 --- [ntainer#0-0-C-1] o.s.k.l.KafkaMessageListenerContainer    : Consumer exception

java.lang.IllegalStateException: This error handler cannot process 'org.apache.kafka.common.KafkaException's; no record information is available
        at org.springframework.kafka.listener.DefaultErrorHandler.handleOtherException(DefaultErrorHandler.java:204) ~[spring-kafka-3.0.1.jar!/:3.0.1]
        at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.handleConsumerException(KafkaMessageListenerContainer.java:1955) ~[spring-kafka-3.0.1.jar!/:3.0.1]
        at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.run(KafkaMessageListenerContainer.java:1390) ~[spring-kafka-3.0.1.jar!/:3.0.1]
        at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804) ~[na:na]
        at java.base/java.lang.Thread.run(Thread.java:831) ~[na:na]
Caused by: org.apache.kafka.common.KafkaException: Received exception when fetching the next record from thetopic. If needed, please seek past the record to continue consumption.
        at org.apache.kafka.clients.consumer.internals.Fetcher$CompletedFetch.fetchRecords(Fetcher.java:1673) ~[kafka-clients-3.3.1.jar!/:na]
        at org.apache.kafka.clients.consumer.internals.Fetcher$CompletedFetch.access$1900(Fetcher.java:1494) ~[kafka-clients-3.3.1.jar!/:na]
        at org.apache.kafka.clients.consumer.internals.Fetcher.fetchRecords(Fetcher.java:716) ~[kafka-clients-3.3.1.jar!/:na]
        at org.apache.kafka.clients.consumer.internals.Fetcher.collectFetch(Fetcher.java:682) ~[kafka-clients-3.3.1.jar!/:na]
        at org.apache.kafka.clients.consumer.KafkaConsumer.pollForFetches(KafkaConsumer.java:1318) ~[kafka-clients-3.3.1.jar!/:na]
        at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1247) ~[kafka-clients-3.3.1.jar!/:na]
        at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1220) ~[kafka-clients-3.3.1.jar!/:na]
        at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.pollConsumer(KafkaMessageListenerContainer.java:1670) ~[spring-kafka-3.0.1.jar!/:3.0.1]
        at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.doPoll(KafkaMessageListenerContainer.java:1645) ~[spring-kafka-3.0.1.jar!/:3.0.1]
        at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.pollAndInvoke(KafkaMessageListenerContainer.java:1446) ~[spring-kafka-3.0.1.jar!/:3.0.1]
        at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.run(KafkaMessageListenerContainer.java:1338) ~[spring-kafka-3.0.1.jar!/:3.0.1]
        ... 2 common frames omitted
Caused by: org.apache.kafka.common.KafkaException: org.xerial.snappy.SnappyError: [UNSUPPORTED_PLATFORM] pure-java snappy requires access to java.nio.Buffer raw address field
        at org.apache.kafka.common.compress.SnappyFactory.wrapForInput(SnappyFactory.java:46) ~[kafka-clients-3.3.1.jar!/:na]
        at org.apache.kafka.common.record.CompressionType$3.wrapForInput(CompressionType.java:94) ~[kafka-clients-3.3.1.jar!/:na]
        at org.apache.kafka.common.record.DefaultRecordBatch.recordInputStream(DefaultRecordBatch.java:276) ~[kafka-clients-3.3.1.jar!/:na]
        at org.apache.kafka.common.record.DefaultRecordBatch.compressedIterator(DefaultRecordBatch.java:280) ~[kafka-clients-3.3.1.jar!/:na]
        at org.apache.kafka.common.record.DefaultRecordBatch.streamingIterator(DefaultRecordBatch.java:364) ~[kafka-clients-3.3.1.jar!/:na]
        at org.apache.kafka.clients.consumer.internals.Fetcher$CompletedFetch.nextFetchedRecord(Fetcher.java:1616) ~[kafka-clients-3.3.1.jar!/:na]
        at org.apache.kafka.clients.consumer.internals.Fetcher$CompletedFetch.fetchRecords(Fetcher.java:1653) ~[kafka-clients-3.3.1.jar!/:na]
        ... 12 common frames omitted
Caused by: org.xerial.snappy.SnappyError: [UNSUPPORTED_PLATFORM] pure-java snappy requires access to java.nio.Buffer raw address field
        at org.xerial.snappy.pure.UnsafeUtil.<clinit>(UnsafeUtil.java:49) ~[snappy-java-1.1.8.4.jar!/:1.1.8.4]
        at org.xerial.snappy.pure.SnappyRawDecompressor.getUnsignedByteSafe(SnappyRawDecompressor.java:323) ~[snappy-java-1.1.8.4.jar!/:1.1.8.4]
        at org.xerial.snappy.pure.SnappyRawDecompressor.readUncompressedLength(SnappyRawDecompressor.java:288) ~[snappy-java-1.1.8.4.jar!/:1.1.8.4]
        at org.xerial.snappy.pure.SnappyRawDecompressor.getUncompressedLength(SnappyRawDecompressor.java:42) ~[snappy-java-1.1.8.4.jar!/:1.1.8.4]
        at org.xerial.snappy.pure.PureJavaSnappy.uncompressedLength(PureJavaSnappy.java:238) ~[snappy-java-1.1.8.4.jar!/:1.1.8.4]
        at org.xerial.snappy.Snappy.uncompressedLength(Snappy.java:638) ~[snappy-java-1.1.8.4.jar!/:1.1.8.4]
        at org.xerial.snappy.SnappyInputStream.readFully(SnappyInputStream.java:145) ~[snappy-java-1.1.8.4.jar!/:1.1.8.4]
        at org.xerial.snappy.SnappyInputStream.readHeader(SnappyInputStream.java:99) ~[snappy-java-1.1.8.4.jar!/:1.1.8.4]
        at org.xerial.snappy.SnappyInputStream.<init>(SnappyInputStream.java:59) ~[snappy-java-1.1.8.4.jar!/:1.1.8.4]
        at org.apache.kafka.common.compress.SnappyFactory.wrapForInput(SnappyFactory.java:44) ~[kafka-clients-3.3.1.jar!/:na]
        ... 18 common frames omitted

已尝试无效的操作 #

手动引入指定版本的snappy-java依赖：

<dependency>
    <groupId>org.xerial.snappy</groupId>
    <artifactId>snappy-java</artifactId>
    <version>1.1.8.4</version>
</dependency>

解决方案 #

方案1：升级snappy-java到兼容JDK17的版本 #

旧版本的纯Java Snappy实现依赖JDK17中被限制的Unsafe API，升级到1.1.10.1及以上版本可解决该问题。修改Maven依赖：

<dependency>
    <groupId>org.xerial.snappy</groupId>
    <artifactId>snappy-java</artifactId>
    <version>1.1.10.1</version>
</dependency>

方案2：添加JVM参数开放模块访问权限 #

在容器启动时添加JVM参数，允许Snappy访问被限制的API：

--add-opens java.base/java.nio=ALL-UNNAMED --add-opens java.base/sun.nio.ch=ALL-UNNAMED

如果使用Dockerfile，可修改ENTRYPOINT或CMD：

ENTRYPOINT ["java", "--add-opens", "java.base/java.nio=ALL-UNNAMED", "--add-opens", "java.base/sun.nio.ch=ALL-UNNAMED", "-jar", "your-app.jar"]

方案3：使用带原生Snappy库的基础镜像 #

选择包含系统级Snappy库的JDK基础镜像，Kafka客户端会优先使用原生库而非纯Java实现，从而绕过Unsafe API的限制。

原因说明 #

JDK17强化了模块系统的访问限制，旧版本snappy-java的纯Java实现依赖的java.nio.Buffer原生地址字段在JDK17中被隐藏，导致无法访问，最终触发UNSUPPORTED_PLATFORM错误。

内容的提问来源于stack exchange，提问作者PatPanda