如何配置JMH（Java微基准测试框架）将输出保存到日志文件？

基于BenchmarkLoop.java示例开发了OrientDB后端的JMH测试，控制台运行正常，但需要将JMH输出保存到日志文件，方便后续评估和长期留存。目前项目已生成target目录，但未找到日志文件，Log4j2也无法捕获JMH的输出内容。

已尝试的解决方案 #

• 网络搜索仅找到《Understanding JMH Output》这类无关内容
• 在Stack Overflow搜索[jmh] output仅得到《JMH: Buggy output in a benchmark》
• 查阅JMH文档、示例及相关建议：
  • CLI方式使用-rf不符合项目可打包的需求
  • resultFormat()方法仅接收ResultFormatType参数，不符合预期

当前控制台输出示例 #

87.039 ms/op

Result "jmh.Test51_1.retrieveASubgraphOf1000Elements":
  97.618 ±(99.9%) 31.200 ms/op [Average]
  (min, avg, max) = (87.039, 97.618, 106.949), stdev = 8.102
  CI (99.9%): [66.418, 128.818] (assumes normal distribution)


# Run complete. Total time: 00:01:42

REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
experiments, perform baseline and negative tests that provide experimental control, make sure
the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
Do not assume the numbers tell you what you want them to tell.

Benchmark                                                     (orientdbUrl)  Mode  Cnt   Score    Error  Units
Test51_1.retrieveASubgraphOf1000Elements  remote:localhost:2424/sparse_data  avgt    5  97.618 ± 31.200  ms/op

Process finished with exit code 0

测试代码 #

package jmh;

import org.apache.commons.configuration2.BaseConfiguration;
import org.apache.commons.configuration2.Configuration;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.apache.tinkerpop.gremlin.orientdb.OrientGraph;
import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversal;
import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversalSource;
import org.apache.tinkerpop.gremlin.structure.Vertex;
import org.openjdk.jmh.annotations.*;
import org.openjdk.jmh.infra.Blackhole;
import org.openjdk.jmh.runner.Runner;
import org.openjdk.jmh.runner.RunnerException;
import org.openjdk.jmh.runner.options.Options;
import org.openjdk.jmh.runner.options.OptionsBuilder;

import java.util.concurrent.TimeUnit;

@BenchmarkMode(Mode.AverageTime)
@OutputTimeUnit(TimeUnit.MILLISECONDS)
@State(Scope.Benchmark)
@Fork(value = 2, jvmArgs = {"-Xms2G", "-Xmx2G"})
public class Test {

    private static final Logger logger = LogManager.getLogger(Test.class);
    @Param("remote:localhost:2424/sparse_data")
    String orientdbUrl;
    Configuration configuration;
    OrientGraph orientGraph;
    static GraphTraversalSource g;

    public static void main(String[] args) throws RunnerException {
        Options opt = new OptionsBuilder()
                .include(Test.class.getSimpleName())
                .forks(1)
                .build();
        new Runner(opt).run();
    }

    @Setup
    public void setup() {
        orientdbUrl = "remote:localhost:2424/sparse_data";
        logger.info("{	orientdb.database.url:	" + orientdbUrl + "	}");
        configuration = new BaseConfiguration();
        configuration.setProperty(OrientGraph.CONFIG_URL, orientdbUrl);
        configuration.setProperty(OrientGraph.CONFIG_USER, "root");
        configuration.setProperty(OrientGraph.CONFIG_PASS, "rootpwd");
        configuration.setProperty(OrientGraph.CONFIG_TRANSACTIONAL, true);
        orientGraph = OrientGraph.open(configuration);
        g = orientGraph.traversal();
    }
    @TearDown
    public void teardown() {
        orientGraph.close();
    }

    @Benchmark
    public static void retrieveASubgraphOf1000Elements() {
        GraphTraversal<Vertex, Vertex> returnValue = g.V().limit(1000);
        while (returnValue.hasNext()) {
            logger.info("returnValue.next():	" + returnValue.next());
        }
    }
}

解决方案 #

方案1：直接生成JMH结果文件（推荐）

通过OptionsBuilder的result()和resultFormat()方法组合配置，在代码层面指定输出文件，无需依赖CLI参数：

修改测试类的main方法：

public static void main(String[] args) throws RunnerException {
    Options opt = new OptionsBuilder()
            .include(Test.class.getSimpleName())
            .forks(1)
            // 指定结果输出路径，target目录下生成日志文件
            .result("target/jmh-benchmark-results.log")
            // 设置输出格式，TEXT与控制台输出一致，可选CSV/JSON等格式
            .resultFormat(ResultFormatType.TEXT)
            .build();
    new Runner(opt).run();
}

方案2：将JMH控制台输出同步到Log4j

Log4j2无法捕获JMH输出是因为JMH直接输出到System.out/System.err，可通过重定向标准流实现同步：

在main方法开头添加以下代码：

import java.io.IOException;
import java.io.OutputStream;
import java.io.PrintStream;

// 重定向System.out到Log4j
System.setOut(new PrintStream(new OutputStream() {
    private final StringBuilder sb = new StringBuilder();
    @Override
    public void write(int b) throws IOException {
        if (b == '\n') {
            logger.info(sb.toString());
            sb.setLength(0);
        } else {
            sb.append((char) b);
        }
    }
}));

// 重定向System.err到Log4j
System.setErr(new PrintStream(new OutputStream() {
    private final StringBuilder sb = new StringBuilder();
    @Override
    public void write(int b) throws IOException {
        if (b == '\n') {
            logger.error(sb.toString());
            sb.setLength(0);
        } else {
            sb.append((char) b);
        }
    }
}));

两种方案可组合使用，既生成独立的JMH结果文件，又能将输出同步到Log4j日志中。

内容的提问来源于stack exchange，提问作者Zach