Kafka ConnectRestExtension部署至plugin.path时无法找到实现类的问题排查求助

大家好，我最近在开发Kafka Connect的Rest扩展时遇到了一个非常奇怪的问题，想请各位帮忙分析排查：

问题现象 #

• 我开发了一个org.apache.kafka.connect.rest.ConnectRestExtension接口的实现类，打包成了排除Connect框架提供依赖的fat jar包
• 当把这个jar包部署到Kafka Connect配置的plugin.path目录下时，启动Connect会抛出以下错误：

  org.apache.kafka.connect.errors.ConnectException: Failed to find any class that implements interface org.apache.kafka.connect.rest.ConnectRestExtension and which name matches~
  

• 但如果把同一个jar包放到Kafka安装目录的libs文件夹（属于Kafka核心类路径）中，Connect却能正常加载并运行这个扩展
• 我已经反复确认过，jar包内resource/META-INF/services/org.apache.kafka.connect.rest.ConnectRestExtension文件里的实现类全限定名是完全正确的

开发依赖与打包配置 #

下面是我开发时使用的Maven依赖和打包配置：

依赖配置 #

<dependencies>
    <!-- connect api -->
    <dependency>
        <groupId>org.apache.kafka</groupId>
        <artifactId>connect-api</artifactId>
        <version>3.5.0</version>
        <scope>provided</scope>
    </dependency>
    <!-- commons -->
    <dependency>
        <groupId>org.apache.commons</groupId>
        <artifactId>commons-lang3</artifactId>
        <version>3.7</version>
    </dependency>
    <dependency>
        <groupId>org.apache.commons</groupId>
        <artifactId>commons-collections4</artifactId>
        <version>4.2</version>
    </dependency>
    <!-- json -->
    <dependency>
        <groupId>com.fasterxml.jackson.core</groupId>
        <artifactId>jackson-core</artifactId>
        <version>${jackson-core.version}</version>
    </dependency>
    <dependency>
        <groupId>com.fasterxml.jackson.core</groupId>
        <artifactId>jackson-databind</artifactId>
        <version>${jackson-core.version}</version>
    </dependency>
    <!-- logging -->
    <dependency>
        <groupId>org.slf4j</groupId>
        <artifactId>slf4j-simple</artifactId>
        <version>1.7.25</version>
    </dependency>
    <!-- okhttp -->
    <dependency>
        <groupId>com.squareup.okhttp3</groupId>
        <artifactId>okhttp</artifactId>
        <version>${okhttp.version}</version>
    </dependency>
    <!-- jersey -->
    <dependency>
        <groupId>org.glassfish.jersey.core</groupId>
        <artifactId>jersey-server</artifactId>
        <version>${jersey.version}</version>
        <scope>provided</scope>
    </dependency>
    <dependency>
        <groupId>org.glassfish.jersey.containers</groupId>
        <artifactId>jersey-container-servlet</artifactId>
        <version>${jersey.version}</version>
        <scope>provided</scope>
    </dependency>
    <dependency>
        <groupId>org.glassfish.jersey.inject</groupId>
        <artifactId>jersey-hk2</artifactId>
        <version>${jersey.version}</version>
        <scope>provided</scope>
    </dependency>
    <dependency>
        <groupId>javax.servlet</groupId>
        <artifactId>javax.servlet-api</artifactId>
        <version>3.1.0</version>
        <scope>provided</scope>
    </dependency>
    <!-- test -->
    <dependency>
        <groupId>org.junit.jupiter</groupId>
        <artifactId>junit-jupiter</artifactId>
        <version>5.8.2</version>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>org.mockito</groupId>
        <artifactId>mockito-core</artifactId>
        <version>5.3.1</version>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>org.mockito</groupId>
        <artifactId>mockito-junit-jupiter</artifactId>
        <version>5.3.1</version>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>com.squareup.okhttp3</groupId>
        <artifactId>mockwebserver</artifactId>
        <version>${okhttp.version}</version>
        <scope>test</scope>
    </dependency>
</dependencies>

打包配置 #

<build>
    <finalName>${project.artifactId}</finalName>
    <resources>
        <resource>
            <directory>src/main/resources</directory>
            <filtering>true</filtering>
        </resource>
    </resources>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-resources-plugin</artifactId>
            <version>2.6</version>
            <configuration>
                <overwrite>true</overwrite>
                <encoding>${source-encoding}</encoding>
            </configuration>
        </plugin>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-shade-plugin</artifactId>
            <version>3.2.4</version>
            <executions>
                <execution>
                    <phase>package</phase>
                    <goals>
                        <goal>shade</goal>
                    </goals>
                    <configuration>
                        <createDependencyReducedPom>false</createDependencyReducedPom>
                    </configuration>
                </execution>
            </executions>
        </plugin>
        <!-- Test coverage report-->
        <plugin>
            <groupId>org.jacoco</groupId>
            <artifactId>jacoco-maven-plugin</artifactId>
            <version>0.8.3</version>
            <executions>
                <execution>
                    <id>default-prepare-agent</id>
                    <goals>
                        <goal>prepare-agent</goal>
                    </goals>
                </execution>
                <execution>
                    <id>default-report</id>
                    <phase>prepare-package</phase>
                    <goals>
                        <goal>report</goal>
                    </goals>
                </execution>
            </executions>
        </plugin>
    </plugins>
</build>
<profiles>
    <!-- app build -->
    <profile>
        <id>app</id>
        <activation>
            <activeByDefault>true</activeByDefault>
        </activation>
    </profile>
</profiles>

日志排查进展 #

我已经跟踪到Kafka Connect加载插件的这段代码，并且能看到日志打印了插件加载位置和注册的类加载器，但还是无法定位到找不到实现类的原因：

private PluginScanResult scanUrlsAndAddPlugins(PluginSource source) {
    log.info("Loading plugin from: {}", source.location());
    if (log.isDebugEnabled()) {
        log.debug("Loading plugin urls: {}", Arrays.toString(source.urls()));
    }
    PluginScanResult plugins = scanPlugins(source);
    log.info("Registered loader: {}", source.loader());
    loadJdbcDrivers(source.loader());
    return plugins;
}

想请教各位，这种现象可能是什么原因导致的？是类加载器隔离的问题？还是打包时的配置遗漏？或者是我对Connect插件加载机制的理解有偏差？

备注：内容来源于stack exchange，提问作者양형욱