JMH基准测试DOM解析器异常结果及相关指标咨询

Hey there! Let's break down your JMH questions one by one—this is a common set of confusions when starting with microbenchmarking DOM parsers, so great questions to ask.

1. Why is the first iteration faster than subsequent ones? #

This seems counterintuitive because JMH warmup iterations usually get faster as the JVM optimizes code, but your scenario flips that. For DOM parsing specifically, here are the most likely culprits:

• FileSystem Cache vs. GC Overhead: If your test parses a local file, the OS caches the file content in memory after the first read. But subsequent iterations create tons of short-lived DOM objects (nodes, elements, etc.), which trigger Minor GC pauses that slow things down. The first iteration runs on an empty heap with no GC pressure, so it feels faster in comparison.
• Parser Internal State Leaks: Some DOM parser implementations don't fully reset their internal state between method calls. Over iterations, leftover objects or configuration could build up, adding unnecessary overhead to subsequent parses.
• JIT Compilation Overhead: Rare, but possible—if the JIT compiler kicks off compilation of helper methods right after the first iteration, the CPU resources used for compiling can make subsequent iterations look slower temporarily.

To debug this, try switching to an in-memory string as your input (to eliminate filesystem cache effects) or explicitly resetting the parser instance inside your benchmark method.

2. What do percentiles and other key metrics mean? #

JMH's output includes several metrics that paint a full picture of your parser's performance:

• Average: The mean time per method call. Useful for high-level performance, but can be skewed by extreme fast/slow calls.
• p0 (Min) / p100 (Max): The fastest and slowest single method call times. These show the absolute bounds of your parser's performance.
• p50 (Median): Half of your method calls are faster than this value, half are slower. This is often more useful than the average because it ignores outliers and reflects typical real-world performance.
• p90, p99, p99.9: These percentiles mean 90%, 99%, or 99.9% of your calls completed faster than the listed time. They're critical for analyzing tail latency—how your parser performs in the worst-case scenarios, which matters a lot for applications requiring consistent response times.
• Error: The statistical margin of error for your results. A smaller error means your benchmark is more reliable and consistent.

For DOM parsing, percentiles are especially valuable because parsing complex vs. simple documents can have huge time differences—median and p99 values will tell you how the parser behaves for most cases, not just an average.

3. Why do results stabilize after the third iteration? #

JMH splits runs into warmup and measurement phases (defaults are 5 warmup, 5 measurement iterations, but you might have adjusted this). The first few iterations are unstable because:

• Class Loading: The JVM has to load all DOM parser, XML processing, and related classes during the first couple of iterations. Once loaded, this overhead disappears.
• JIT Compilation: The JVM's Just-In-Time compiler only optimizes methods after they've been called multiple times. After 2-3 iterations, your core benchmark method and its dependencies will be compiled to optimized machine code (C1/C2 levels), so performance stops improving.
• Cache Warmup: CPU instruction caches, data caches, and even JVM-level caches need a few iterations to "warm up"—once they're populated, the CPU can execute code much more efficiently.

Once all these initialization steps are done, the execution environment is consistent across iterations, so your results stabilize.

4. Does one iteration equal one execution of the benchmark method? #

No—this is a super common misunderstanding. JMH iterations are batches of method calls, not single executions:

• By default, JMH uses time-based iterations: each iteration runs for a fixed duration (e.g., 1 second), and JMH calls your benchmark method as many times as possible within that window. Metrics are calculated across all those calls.
• You can also configure count-based iterations, where each iteration runs a fixed number of method calls.

To see exactly how many times your method is invoked per iteration, run JMH with the -f 1 -v flags—this verbose output will show you invocation counts, along with other useful details about the benchmark run.

内容的提问来源于stack exchange，提问作者Tri Nguyen