启用HA的Flink独立应用启动失败：QuorumMaj方法缺失

问题现象 #

部署基于ZooKeeper的Flink 1.15.2 StandaloneApplication高可用模式时，JobManager和TaskManager初始化均失败，核心报错如下：

[2022-09-07 09:09:28,871] ERROR [org.apache.flink.runtime.entrypoint.ClusterEntrypoint] - Could not start cluster entrypoint StandaloneApplicationClusterEntryPoint.
org.apache.flink.runtime.entrypoint.ClusterEntrypointException: Failed to initialize the cluster entrypoint StandaloneApplicationClusterEntryPoint.
    at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:250) ~[jobs-0.1.1-20220907.080009-87.jar:?]
    at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runClusterEntrypoint(ClusterEntrypoint.java:711) [jobs-0.1.1-20220907.080009-87.jar:?]
    at org.apache.flink.container.entrypoint.StandaloneApplicationClusterEntryPoint.main(StandaloneApplicationClusterEntryPoint.java:82) [jobs-0.1.1-20220907.080009-87.jar:?]
Caused by: java.lang.NoSuchMethodError: 'void org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.server.quorum.flexible.QuorumMaj.<init>(java.util.Map)'
    at org.apache.flink.shaded.curator5.org.apache.curator.framework.imps.EnsembleTracker.<init>(EnsembleTracker.java:57) ~[jobs-0.1.1-20220907.080009-87.jar:?]
    at org.apache.flink.shaded.curator5.org.apache.curator.framework.imps.CuratorFrameworkImpl.<init>(CuratorFrameworkImpl.java:187) ~[jobs-0.1.1-20220907.080009-87.jar:?]
    at org.apache.flink.shaded.curator5.org.apache.curator.framework.CuratorFrameworkFactory$Builder.build(CuratorFrameworkFactory.java:188) ~[jobs-0.1.1-20220907.080009-87.jar:?]
    at org.apache.flink.runtime.util.ZooKeeperUtils.startCuratorFramework(ZooKeeperUtils.java:289) ~[jobs-0.1.1-20220907.080009-87.jar:?]
    at org.apache.flink.runtime.util.ZooKeeperUtils.startCuratorFramework(ZooKeeperUtils.java:274) ~[jobs-0.1.1-20220907.080009-87.jar:?]
    at org.apache.flink.runtime.highavailability.HighAvailabilityServicesUtils.createZooKeeperHaServices(HighAvailabilityServicesUtils.java:90) ~[jobs-0.1.1-20220907.080009-87.jar:?]
    at org.apache.flink.runtime.highavailability.HighAvailabilityServicesUtils.createHighAvailabilityServices(HighAvailabilityServicesUtils.java:140) ~[jobs-0.1.1-20220907.080009-87.jar:?]
    at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.createHaServices(ClusterEntrypoint.java:427) ~[jobs-0.1.1-20220907.080009-87.jar:?]
    at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.initializeServices(ClusterEntrypoint.java:376) ~[jobs-0.1.1-20220907.080009-87.jar:?]
    at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runCluster(ClusterEntrypoint.java:277) ~[jobs-0.1.1-20220907.080009-87.jar:?]
    at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.lambda$startCluster$1(ClusterEntrypoint.java:227) ~[jobs-0.1.1-20220907.080009-87.jar:?]
    at java.security.AccessController.doPrivileged(Native Method) ~[?:?]

环境与依赖配置 #

使用Java 11 + Maven构建胖jar，依赖Flink 1.15.2，核心pom依赖片段如下：

<...>
        <dependency>
            <groupId>org.apache.flink</groupId>
            <artifactId>flink-java</artifactId>
            <version>${flink.version}</version>
            <scope>compile</scope>
        </dependency>
        <dependency>
            <groupId>org.apache.flink</groupId>
            <artifactId>flink-core</artifactId>
            <version>${flink.version}</version>
            <scope>compile</scope>
        </dependency>
        <dependency>
            <groupId>org.apache.flink</groupId>
            <artifactId>flink-streaming-java</artifactId>
            <version>${flink.version}</version>
            <scope>compile</scope>
        </dependency>
        <dependency>
            <groupId>org.apache.flink</groupId>
            <artifactId>flink-avro</artifactId>
            <version>${flink.version}</version>
            <scope>compile</scope>
        </dependency>
        <dependency>
            <groupId>org.apache.flink</groupId>
            <artifactId>flink-connector-kafka</artifactId>
            <version>${flink.version}</version>
        </dependency>
        <dependency>
            <groupId>org.apache.flink</groupId>
            <artifactId>flink-container</artifactId>
            <version>${flink.version}</version>
        </dependency>
        <dependency>
            <groupId>org.apache.flink</groupId>
            <artifactId>flink-clients</artifactId>
            <version>${flink.version}</version>
        </dependency>
        <dependency>
            <groupId>org.apache.flink</groupId>
            <artifactId>flink-runtime</artifactId>
            <version>${flink.version}</version>
            <scope>compile</scope>
        </dependency>
        <dependency>
            <groupId>org.apache.flink</groupId>
            <artifactId>flink-metrics-core</artifactId>
            <version>${flink.version}</version>
            <scope>compile</scope>
        </dependency>
        <dependency>
            <groupId>org.apache.flink</groupId>
            <artifactId>flink-shaded-zookeeper-3</artifactId>
            <version>3.6.3-15.0</version>
            <scope>compile</scope>
        </dependency>
<...>

已尝试操作：添加shaded curator依赖并加入类路径、更换ZooKeeper 3.4/3.5版本，均未解决问题。

问题根源 #

这是Flink shaded组件版本不兼容导致的：
Flink 1.15.x的flink-runtime已间接依赖了匹配版本的flink-shaded-zookeeper-3和flink-shaded-curator5，手动引入的ZooKeeper shaded包会打破依赖匹配关系，Curator5调用的QuorumMaj构造方法在冲突的ZooKeeper包中不存在，从而抛出NoSuchMethodError。

解决思路 #

• 移除手动引入的flink-shaded-zookeeper-3依赖
  Flink runtime已自带适配的shaded ZooKeeper包，手动引入会引发依赖冲突，直接删除pom中的该依赖即可。

• 配置Maven Shade插件过滤冲突依赖
  构建胖jar时，通过插件确保仅保留Flink官方shaded的ZooKeeper/Curator组件，避免原生依赖混入：

  <plugin>
      <groupId>org.apache.maven.plugins</groupId>
      <artifactId>maven-shade-plugin</artifactId>
      <version>3.2.4</version>
      <executions>
          <execution>
              <phase>package</phase>
              <goals>
                  <goal>shade</goal>
              </goals>
              <configuration>
                  <relocations>
                      <relocation>
                          <pattern>org.apache.flink.shaded</pattern>
                          <shadedPattern>org.apache.flink.shaded</shadedPattern>
                      </relocation>
                  </relocations>
                  <filters>
                      <filter>
                          <artifact>*:*</artifact>
                          <excludes>
                              <exclude>org.apache.zookeeper:*</exclude>
                              <exclude>org.apache.curator:*</exclude>
                          </excludes>
                      </filter>
                  </filters>
                  <transformers>
                      <transformer implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer"/>
                  </transformers>
              </configuration>
          </execution>
      </executions>
  </plugin>
  

• 验证ZooKeeper集群版本兼容性
  Flink 1.15.x适配的ZooKeeper版本为3.6.3，集群版本需为3.4.x及以上（推荐3.6.x），版本差距过大也会引发兼容性问题。

• 检查类路径是否存在冲突
  构建完成后，执行jar -tf your-fatjar.jar | grep zookeeper，确保仅存在org.apache.flink.shaded.zookeeper3相关类，无原生org.apache.zookeeper类。

内容的提问来源于stack exchange，提问作者Dimitrios