部署Flink应用至AWS时遇NoSuchMethodError问题求助

错误信息 #

java.lang.NoSuchMethodError: 'org.apache.flink.metrics.MetricGroup org.apache.flink.api.common.functions.RuntimeContext.getMetricGroup()'
at software.amazon.kinesis.connectors.flink.internals.KinesisDataFetcher.<init>(KinesisDataFetcher.java:429)
at software.amazon.kinesis.connectors.flink.internals.KinesisDataFetcher.<init>(KinesisDataFetcher.java:380)
at software.amazon.kinesis.connectors.flink.FlinkKinesisConsumer.createFetcher(FlinkKinesisConsumer.java:473)
at software.amazon.kinesis.connectors.flink.FlinkKinesisConsumer.run(FlinkKinesisConsumer.java:289)
at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:110)
at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:67)
at org.apache.flink.streaming.runtime.tasks.SourceStreamTask$LegacySourceFunctionThread.run(SourceStreamTask.java:332)

当前依赖配置 #

dependencies {
    compileOnly('org.projectlombok:lombok:1.18.26')
    annotationProcessor('org.projectlombok:lombok:1.18.26')
    implementation('com.amazonaws:aws-lambda-java-core:1.2.2')
    implementation('org.apache.logging.log4j:log4j-core:2.20.0')
    implementation('com.fasterxml.jackson.core:jackson-databind:2.15.1')
    implementation("org.apache.flink:flink-connector-kinesis:1.15.4")
    compileOnly('org.apache.flink:flink-streaming-java:1.15.4')
    implementation('com.amazonaws:aws-java-sdk-lambda:1.12.472')
    implementation('software.amazon.kinesis:amazon-kinesis-connector-flink:2.3.0')
    implementation('com.amazonaws:aws-kinesisanalytics-flink:2.0.0')
    implementation("com.amazonaws:aws-kinesisanalytics-runtime:1.2.0")
    implementation('com.amazonaws:aws-java-sdk-appconfig:1.12.471')
    implementation("software.amazon.awssdk:appconfig:2.20.121")

    testImplementation('org.mockito:mockito-core:2.22.0')
    testImplementation('org.mockito:mockito-inline:3.4.0')
    testImplementation('org.junit.jupiter:junit-jupiter:5.7.1')
    testImplementation("org.apache.flink:flink-test-utils_2.12:1.11.1")
    testImplementation("org.powermock:powermock-api-mockito2:2.0.9")
    testImplementation("org.powermock:powermock-module-junit4:2.0.9")
    testImplementation('org.apache.flink:flink-streaming-java_2.12:1.14.4:tests')
    testImplementation('org.apache.flink:flink-runtime_2.12:1.13.6')
    compileOnly('org.projectlombok:lombok:1.18.26')
    annotationProcessor("org.projectlombok:lombok:1.18.22")
}

问题排查与解决方案 #

这个错误的核心原因是Flink核心类版本冲突：RuntimeContext.getMetricGroup()方法在不同Flink版本中存在签名差异，编译时依赖的新版本类（包含该方法），但运行时加载了旧版本类（缺少该方法），导致方法找不到。

具体问题点 #

1. Kinesis连接器冲突：同时引入了官方Flink连接器flink-connector-kinesis:1.15.4和AWS的amazon-kinesis-connector-flink:2.3.0，两者可能依赖不同Flink版本，引发类加载冲突。
2. 测试依赖版本混乱：测试用的Flink依赖版本不统一（1.11.1、1.13.6、1.14.4），这些旧版本的RuntimeContext类会混入运行时环境，覆盖主依赖的新版本类。
3. 重复依赖冗余：存在重复的Lombok依赖（版本不一致），可能间接影响依赖解析。

修复步骤 #

1. 统一Flink核心版本：所有Flink相关依赖（包括测试）统一使用1.15.4版本，确保编译和运行时类一致。
2. 清理冲突的Kinesis连接器：二选一保留官方或AWS的Kinesis连接器，推荐使用与Flink版本匹配的AWS连接器（amazon-kinesis-connector-flink:2.3.0适配Flink 1.15.x），移除官方的flink-connector-kinesis。
3. 修复测试依赖：替换所有旧版本的Flink测试依赖为1.15.4。
4. 清理重复依赖：统一Lombok版本，移除重复配置。

修改后的依赖示例 #

dependencies {
    compileOnly('org.projectlombok:lombok:1.18.26')
    annotationProcessor('org.projectlombok:lombok:1.18.26')
    
    implementation('com.amazonaws:aws-lambda-java-core:1.2.2')
    implementation('org.apache.logging.log4j:log4j-core:2.20.0')
    implementation('com.fasterxml.jackson.core:jackson-databind:2.15.1')
    // 保留AWS Kinesis连接器，移除官方连接器避免冲突
    implementation('software.amazon.kinesis:amazon-kinesis-connector-flink:2.3.0')
    implementation('com.amazonaws:aws-kinesisanalytics-flink:2.0.0')
    implementation("com.amazonaws:aws-kinesisanalytics-runtime:1.2.0")
    implementation('com.amazonaws:aws-java-sdk-lambda:1.12.472')
    implementation('com.amazonaws:aws-java-sdk-appconfig:1.12.471')
    implementation("software.amazon.awssdk:appconfig:2.20.121")
    
    // 主Flink依赖保持1.15.4
    compileOnly('org.apache.flink:flink-streaming-java:1.15.4')

    testImplementation('org.mockito:mockito-core:2.22.0')
    testImplementation('org.mockito:mockito-inline:3.4.0')
    testImplementation('org.junit.jupiter:junit-jupiter:5.7.1')
    testImplementation("org.powermock:powermock-api-mockito2:2.0.9")
    testImplementation("org.powermock:powermock-module-junit4:2.0.9")
    // 统一测试用Flink版本为1.15.4
    testImplementation("org.apache.flink:flink-test-utils_2.12:1.15.4")
    testImplementation('org.apache.flink:flink-streaming-java_2.12:1.15.4:tests')
    testImplementation('org.apache.flink:flink-runtime_2.12:1.15.4')
}

额外验证 #

执行gradle dependencies命令查看依赖树，确认所有Flink相关依赖的版本都是1.15.4，若存在旧版本传递依赖，使用exclude配置手动移除。

内容的提问来源于stack exchange，提问作者Dipesh Darji