TestNG含DataProvider的测试在Jenkins上无法运行求助

环境信息 #

• Maven 3.9.2（Jenkins使用Docker镜像maven:3.9.2-eclipse-temurin-11）
• JDK 11（Jenkins使用Docker镜像maven:3.9.2-eclipse-temurin-11）
• TestNG 7.8.0

执行命令 #

mvn -Ddependency.surefire.verbose=10 -Pcoverage verify -B -U

相关POM配置片段 #

<build>
    <pluginManagement>
      <plugins>
        <plugin>
          <groupId>org.jacoco</groupId>
          <artifactId>jacoco-maven-plugin</artifactId>
          <version>0.8.10</version>
        </plugin>
        ...
      </plugins>
    </pluginManagement>
...
</build>
...
<profiles>
    <profile>
      <id>coverage</id>
      <build>
        <plugins>
          <plugin>
            <groupId>org.jacoco</groupId>
            <artifactId>jacoco-maven-plugin</artifactId>
            <configuration>
              <excludes>
                <exclude>com/XXX/framework/dataproviders/**/*</exclude>
                <exclude>com/XXX/v2/basetest/**/*</exclude>
                <exclude>**/*Exception*</exclude>
              </excludes>
            </configuration>
            <executions>
              <execution>
                <goals>
                  <goal>prepare-agent</goal>
                </goals>
              </execution>
              <!-- attached to Maven test phase -->
              <execution>
                <id>report</id>
                <phase>test</phase>
                <goals>
                  <goal>report</goal>
                </goals>
              </execution>
              <!-- Add this checking -->
              <execution>
                <id>jacoco-check</id>
                <goals>
                  <goal>check</goal>
                </goals>
                <configuration>
                  <rules>
                    <rule>
                      <element>BUNDLE</element>
                      <limits>
                        <limit>
                          <counter>INSTRUCTION</counter>
                          <value>COVEREDRATIO</value>
                          <minimum>60%</minimum>
                        </limit>
                        <limit>
                          <counter>CLASS</counter>
                          <value>MISSEDCOUNT</value>
                          <maximum>0</maximum>
                        </limit>
                      </limits>
                    </rule>
                    <rule>
                      <element>PACKAGE</element>
                      <limits>
                        <limit>
                          <counter>LINE</counter>
                          <value>COVEREDRATIO</value>
                          <minimum>60%</minimum>
                        </limit>
                      </limits>
                    </rule>
                  </rules>
                  <skip>${skipTests}</skip>
                </configuration>
              </execution>
            </executions>
          </plugin>
        </plugins>
      </build>
    </profile>

问题现象 #

含DataProvider的测试在本地可正常执行，但Jenkins上无任何执行记录。

本地日志片段 #

[2023-06-06 16:55:46.795] [INFO] [TestClass] Creating TestClass for [ClassImpl class=com.XXX.framework.utils.TimeUtilsTest]
[2023-06-06 16:55:46.796] [INFO] Method public java.lang.Object[][] com.XXX.framework.utils.TimeUtilsTest.getTimeData() has a @Test annotation but also a return value: ignoring it. Use <suite allow-return-values="true"> to fix this
[2023-06-06 16:55:46.797] [INFO] [TestClass] Adding method TimeUtilsTest.testToFormattedString(java.util.concurrent.TimeUnit,int,java.lang.String,com.XXX.framework.annotations.TestNameParameter)[pri:0, instance:null] on TestClass class com.XXX.framework.utils.TimeUtilsTest

...

[2023-06-06 16:55:47.316] [INFO] ===== Test class
com.XXX.framework.utils.TimeUtilsTest
[2023-06-06 16:55:47.316] [INFO]     @Test TimeUtilsTest.testToFormattedString(java.util.concurrent.TimeUnit,int,java.lang.String,com.XXX.framework.annotations.TestNameParameter)[pri:0, instance:com.XXX.framework.utils.TimeUtilsTest@283eb984]
[2023-06-06 16:55:47.316] [INFO] ======

...

[TestNG] INVOKING: "Surefire test" - com.XXX.framework.utils.TimeUtilsTest.testToFormattedString(java.util.concurrent.TimeUnit,int,java.lang.String,com.XXX.framework.annotations.TestNameParameter)(value(s): NANOSECONDS, 60, "60ns", TestNameParameter(customName=60 NANOSECONDS))

测试可正常执行并输出结果。

Jenkins日志片段 #

[2023-06-06 21:02:29.270] [INFO] [TestClass] Creating TestClass for [ClassImpl class=com.XXX.framework.utils.TimeUtilsTest]
[2023-06-06 21:02:29.271] [INFO] Method public java.lang.Object[][] com.XXX.framework.utils.TimeUtilsTest.getTimeData() has a @Test annotation but also a return value: ignoring it. Use <suite allow-return-values="true"> to fix this
[2023-06-06 21:02:29.271] [INFO] [TestClass] Adding method TimeUtilsTest.testToFormattedString(java.util.concurrent.TimeUnit,int,java.lang.String,com.XXX.framework.annotations.TestNameParameter)[pri:0, instance:null] on TestClass class com.XXX.framework.utils.TimeUtilsTest

...

[2023-06-06 21:02:29.529] [INFO] ===== Test class
com.XXX.framework.utils.TimeUtilsTest
[2023-06-06 21:02:29.529] [INFO]     @Test TimeUtilsTest.testToFormattedString(java.util.concurrent.TimeUnit,int,java.lang.String,com.XXX.framework.annotations.TestNameParameter)[pri:0, instance:com.XXX.framework.utils.TimeUtilsTest@52f9a620]
[2023-06-06 21:02:29.529] [INFO] ======

后续日志中无该测试类的其他记录，所有含DataProvider的测试均无法运行。

已尝试的操作 #

• 确认使用正确的@Test注解（TestNG而非JUnit）
• 移除测试分组或可能干扰的监听器
• 降级TestNG版本
• 祈祷（无效）

排查方向建议 #

1. 修正DataProvider方法注解：日志明确提示getTimeData()方法被错误标记了@Test注解，虽然本地能忽略该问题执行，但Jenkins环境可能因逻辑差异直接跳过测试。建议将该方法的@Test注解替换为@DataProvider注解。
2. 对齐Surefire插件版本：检查本地与Jenkins环境的maven-surefire-plugin版本是否一致，项目POM或Docker镜像内置的版本差异可能导致测试执行逻辑不同。
3. 导出TestNG执行套件对比：在Jenkins执行命令时添加-Dtestng.dumpXml=true参数，导出实际执行的套件XML，与本地生成的文件对比，确认DataProvider测试是否被过滤。
4. 排除Jacoco干扰：在Jenkins上执行不带-Pcoverage参数的测试，验证是否Jacoco的prepare-agent目标导致TestNG执行异常。
5. 排查依赖冲突：在Jenkins上执行mvn dependency:tree，对比本地依赖树，检查是否存在TestNG或Surefire相关的依赖冲突。
6. 提升TestNG日志级别：添加-Dorg.testng.logging.level=DEBUG参数，获取更详细的TestNG执行日志，定位测试被跳过的具体原因。

内容的提问来源于stack exchange，提问作者javydreamercsw