使用Scala+Cloud Dataflow对接Firestore时java.lang.VerifyError排查

我用Cloud Dataflow将数据写入Firestore，本地运行完全正常，但打包成Docker镜像提交到云端时出现以下类验证错误，不清楚问题出在哪。

错误信息：

java.lang.VerifyError: Bad type on operand stack Exception Details: Location: com/google/firestore/v1/Value$Builder.setTimestampValue(Lcom/google/protobuf/Timestamp;)Lcom/google/firestore/v1/Value$Builder; @36: invokevirtual Reason: Type 'com/google/protobuf/Timestamp' (current frame, stack[1]) is not assignable to 'com/google/protobuf/AbstractMessage' Current Frame: bci: @36 flags: { } locals: { 'com/google/firestore/v1/Value$Builder', 'com/google/protobuf/Timestamp' } stack: { 'com/google/protobuf/SingleFieldBuilderV3', 'com/google/protobuf/Timestamp' } Bytecode: 0x0000000: 2ab4 0015 c700 1b2b c700 0bbb 0049 59b7 0x0000010: 004a bf2a 2bb5 000e 2ab6 0032 a700 0c2a 0x0000020: b400 152b b600 5857 2a10 0ab5 0009 2ab0 0x0000030: Stackmap Table: same_frame(@19) same_frame(@31) same_frame(@40)

build.sbt配置 #

val scioVersion = "0.6.1"
val beamVersion = "2.6.0"
val scalaMacrosVersion = "2.1.1"

unmanagedBase := baseDirectory.value / "lib"

lazy val root: Project = project
  .in(file("."))
  .settings(commonSettings)
  .settings(macroSettings)
  .settings(
    name := "scio-job",
    description := "scio job",
    publish / skip := true,
    libraryDependencies ++= Seq(
      "com.spotify" %% "scio-core" % scioVersion,
      "com.spotify" %% "scio-extra" % scioVersion,
      "com.spotify" %% "scio-test" % scioVersion % Test,
      "org.apache.beam" % "beam-runners-direct-java" % beamVersion,
      // optional dataflow runner
       "org.apache.beam" % "beam-runners-google-cloud-dataflow-java" % beamVersion,
      "org.slf4j" % "slf4j-simple" % "1.7.25",
      "com.nrinaudo" %% "kantan.csv" % "0.4.0",
      "com.github.nscala-time" %% "nscala-time" % "2.20.0",
      "com.google.cloud" % "google-cloud-storage" % "1.49.0",
      "io.circe" %% "circe-parser" % "0.10.0",
      "io.circe" %% "circe-optics" % "0.10.0",
      "org.json4s" %% "json4s-jackson" % "3.6.6",

      "me.lyh" %% "shapeless-datatype-datastore_1.3" % "0.1.10",
      "com.google.cloud" % "google-cloud-bigquery" % "1.35.0",
      "com.google.cloud" % "google-cloud-secretmanager" % "1.4.2",
      "io.grpc" % "grpc-okhttp" % "1.24.0",
      "com.fasterxml.jackson.module" %% "jackson-module-scala" % "2.12.2",
      "com.google.cloud" % "google-cloud-secretmanager" % "1.4.2",
      "org.springframework.boot" % "spring-boot-starter-data-jpa" % "2.0.4.RELEASE",
      "com.google.cloud" % "google-cloud-firestore" % "3.11.0"
    )
  )
  .enablePlugins(PackPlugin, sbtdocker.DockerPlugin)

Dockerfile配置 #

FROM openjdk:8-jre-stretch
WORKDIR /script
ADD ./target/pack .
ADD ./script .
USER root
RUN chmod 777 -R /script
RUN echo "deb http://archive.debian.org/debian stretch main contrib non-free" > /etc/apt/sources.list
RUN apt-get update
RUN apt-get install lsb-release -y
RUN apt-get install -my wget gnupg
RUN ln -s $JAVA_HOME/lib $JAVA_HOME/conf
RUN export CLOUD_SDK_REPO="cloud-sdk-$(lsb_release -c -s)" && \
    echo "deb http://packages.cloud.google.com/apt $CLOUD_SDK_REPO main" | tee -a /etc/apt/sources.list.d/google-cloud-sdk.list  && \
    curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add - && \
    apt-get update -y && apt-get install google-cloud-sdk -y

问题分析与解决方案 #

这个VerifyError是典型的依赖版本冲突导致的类定义不兼容问题，核心原因是Protobuf相关类在运行时出现版本不匹配：

• 本地环境依赖版本统一，而云端Docker环境中可能混入了不同版本的Protobuf库，导致JVM加载类时验证失败。
• 当前使用的Beam 2.6.0版本较旧，与google-cloud-firestore 3.11.0依赖的Protobuf版本存在兼容性问题。

具体解决步骤：

1. 统一Protobuf版本：
   在build.sbt中添加强制统一的Protobuf版本配置，先执行sbt dependencyTree查看当前依赖树，确认冲突的Protobuf版本，再选择合适的统一版本，示例：

   libraryDependencies ++= Seq(
     // 其他依赖...
   ).map(_.exclude("com.google.protobuf", "protobuf-java"))
   libraryDependencies += "com.google.protobuf" % "protobuf-java" % "3.19.4"
   

2. 升级Scio和Beam版本：
   Scio 0.6.1和Beam 2.6.0都是多年前的版本，建议升级到稳定的新版本，例如Scio 0.12.x搭配Beam 2.40.x，新版本会修复大量依赖兼容性问题。

3. 优化Docker打包：

   • 确保Docker镜像中只包含打包后的必要依赖，避免冗余库干扰。
   • 可以使用sbt-assembly插件打包成单一jar包，减少依赖冲突的概率，替代PackPlugin。

4. 清理依赖缓存：
   本地执行sbt clean清理缓存后重新打包，确保Docker镜像中的依赖是最新且一致的。

内容的提问来源于stack exchange，提问作者thayanee ruennark