通过CLI与IntelliJ运行Java Jar时的测试执行不一致问题

项目结构 #

src
└── main
    └── java
        └── tests
            ├── functional
            │   ├── TestA
            │   └── TestB
            ├── verification
            │   └── TestC
            └── TestsRunner.java
target
├── classes
├── lib
│   ├── depA
│   ├── depB
│   └── ..
├── app.jar
└── ..

主类TestsRunner.java代码 #

package tests;

import lombok.extern.slf4j.Slf4j;
import picocli.CommandLine;

import java.util.concurrent.Callable;


@Slf4j
@CommandLine.Command(name = "run", mixinStandardHelpOptions = true, version = "1.0", description = "Run tests")
public class TestsRunner implements Callable<Integer> {

  @CommandLine.Option(names = {"-s", "--suite"}, required = true, description = "Suite name")
  private String testSuiteName;

  @CommandLine.Option(names = {"-o", "--output"}, required = true, description = "Test reports output directory")
  private String reportOutputDir;

  public static void main(String[] args) {
    int exitCode = new CommandLine(new TestsRunner()).execute(args);
    System.exit(exitCode);
  }

  @Override
  public Integer call() throws Exception {
    var basePackage = switch (testSuiteName) {
      case "functional" -> "tests.functional";
      case "verification" -> "tests.verification";
      default -> throw new RuntimeException("Unknown tests suite: " + testSuiteName);
    };
    var runner = Junit5ServiceTestsRunner.builder()
      .classNamePattern(".*Test")
      .classNamePattern(".*Tests")
      .basePackage(basePackage)
      .reportOutputDir(reportOutputDir)
      .build();
    var result = runner.run();
    return result.isSuccess() ? 0 : 1;
  }
}

Maven jar插件配置 #

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-jar-plugin</artifactId>
    <version>3.2.0</version>
    <configuration>
        <finalName>app</finalName>
        <archive>
            <manifest>
                <addClasspath>true</addClasspath>
                <Main-Class>tests.TestsRunner</Main-Class>
                <classpathPrefix>lib</classpathPrefix>
                <!--workaround for MJAR-156-->
                <useUniqueVersions>false</useUniqueVersions>
            </manifest>
            <manifestEntries>
                <Class-Path>.</Class-Path>
            </manifestEntries>
        </archive>
    </configuration>
</plugin>

运行情况对比 #

IntelliJ中运行正常 #

通过「Modify run configurations」传入-s verification -o report参数，能正常执行测试，输出：

Test run finished after 4246 ms
[         3 containers found      ]
[         0 containers skipped    ]
[         3 containers started    ]
[         0 containers aborted    ]
[         3 containers successful ]
[         0 containers failed     ]
[         1 tests found           ]
[         0 tests skipped         ]
[         1 tests started         ]
[         0 tests aborted         ]
[         1 tests successful      ]
[         0 tests failed          ]

IntelliJ的运行命令（类路径包含项目classes目录和所有依赖）：

"C:\Program Files\Java\jdk-17.0.8\bin\java.exe" "-javaagent:C:\Program Files\JetBrains\IntelliJ IDEA Community Edition 2022.1.3\lib\idea_rt.jar=65474:C:\Program Files\JetBrains\IntelliJ IDEA Community Edition 2022.1.3\bin" -Dfile.encoding=UTF-8 -classpath C:\Users\xxx\Documents\GitHub\my-project\tests\target\classes;C:\Users\xxx\.m2\repository\info\picocli\picocli\4.7.1\picocli-4.7.1.jar;C:\Users\xxx\.m2\repository\com\core\testing\mock-servers-junit5\1.1.0\mock-servers-junit5-1.1.0.jar ..........(all other dependencies) tests.TestsRunner -s verification -o report

命令行运行异常 #

在target目录执行：

java -jar app.jar -o ${JUNIT_OUTPUT_FOLDER} -s ${SUITE_NAME}

或者尝试指定类路径：

java -cp "app.jar:lib/*:classes" tests.TestsRunner -s verification -o reports

均无法识别测试用例，输出：

Test run finished after 62 ms
[         1 containers found      ]
[         0 containers skipped    ]
[         1 containers started    ]
[         0 containers aborted    ]
[         1 containers successful ]
[         0 containers failed     ]
[         0 tests found           ]
[         0 tests skipped         ]
[         0 tests started         ]
[         0 tests aborted         ]
[         0 tests successful      ]
[         0 tests failed          ]

说明：测试用例使用JUnit5的@TestTemplate注解和扩展类触发，没有普通@Test注解的方法；项目由模板生成，同结构其他项目无此问题。已检查app.jar的Manifest配置正常，依赖、主类、测试类均存在。

解决建议 #

• 检查测试类是否被打包进app.jar：解压app.jar，确认tests/functional/TestA.class、tests/verification/TestC.class等测试类是否存在。如果缺失，说明Maven打包时未包含测试类，需调整打包配置（比如用maven-assembly-plugin/shade-plugin，或确保测试类编译后被纳入jar）。
• 调整类路径顺序：命令行运行时将classes目录放在类路径最前面，优先加载本地编译的测试类：

  # Windows系统
  java -cp "classes;app.jar;lib/*" tests.TestsRunner -s verification -o reports
  # Linux/macOS系统
  java -cp "classes:app.jar:lib/*" tests.TestsRunner -s verification -o reports
  

• 排查自定义Runner的扫描逻辑：
  1. 确认classNamePattern是否正确匹配测试类名称，可临时添加日志打印扫描到的类列表；
  2. 检查basePackage参数是否正确传递，在call()方法中打印该参数值验证；
  3. 确认Junit5ServiceTestsRunner支持@TestTemplate注解和扩展类，部分自定义Runner需额外配置才能识别这类测试。
• 检查依赖冲突：执行mvn dependency:tree查看依赖树，对比IntelliJ运行时的依赖版本，排除JUnit5相关依赖的版本冲突。
• 添加日志定位问题：在call()方法中添加日志，打印Runner的配置参数、扫描结果等信息，帮助排查扫描过程中的问题：

  @Override
  public Integer call() throws Exception {
    var basePackage = switch (testSuiteName) {
      case "functional" -> "tests.functional";
      case "verification" -> "tests.verification";
      default -> throw new RuntimeException("Unknown tests suite: " + testSuiteName);
    };
    log.info("Scanning base package: {}", basePackage);
    var runner = Junit5ServiceTestsRunner.builder()
      .classNamePattern(".*Test")
      .classNamePattern(".*Tests")
      .basePackage(basePackage)
      .reportOutputDir(reportOutputDir)
      .build();
    log.info("Runner configuration details: {}", runner.toString());
    var result = runner.run();
    log.info("Final test result: {}", result);
    return result.isSuccess() ? 0 : 1;
  }
  

内容的提问来源于stack exchange，提问作者RanAbitbul