JDK8u151升级至u161/u171后Tomcat POST接口吞吐量大幅下降求助

Hey there, let's break down this throughput drop you're experiencing when switching your Tomcat POST service from JDK 8u151 to 8u161/u171. I’ve debugged similar JDK version-related performance hits before, so here’s a practical, step-by-step approach to get to the root of it:

First, let’s recap the impactful updates between these versions that often hit web services:

• GC Tuning Defaults: Oracle adjusted default parameters for ParallelGC (the default collector for server-class machines) in u161, including tweaks to heap allocation and metaspace sizing that can increase GC pause frequency.
• SSL/TLS Updates: JDK 8u161 enabled TLS 1.3 by default for compatible environments, and updated default cipher suites—this can add overhead if your Tomcat setup isn’t optimized for the new protocols.
• Security Policy Hardening: Minor changes to permission checks for network I/O and classloading can introduce unexpected latency in high-throughput scenarios.
• Network Stack Adjustments: Under-the-hood changes to the NIO implementation (used by Tomcat’s default connector) might alter how connections are handled, especially for POST requests with large payloads.

Let’s narrow down the issue with actionable tests:

1. Compare Default JVM Parameters
   Run this command on both JDK versions to spot differences:

   java -XX:+PrintFlagsFinal -version > jvm-flags-[version].txt
   

   Focus on parameters related to heap size (InitialHeapSize, MaxHeapSize), metaspace (MetaspaceSize), and GC threads (ParallelGCThreads). If u161/u171 uses smaller default heap/metaspace, that could trigger more frequent garbage collection.

2. Analyze GC Behavior
   Enable detailed GC logging for both JDK versions by adding these flags to your Tomcat startup script:

   -Xloggc:/path/to/gc.log -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintGCApplicationStoppedTime
   

   Use a tool like GCViewer to compare logs. Look for longer stop-the-world (STW) pauses or more frequent Full GCs in the newer JDK versions—this is a common culprit for throughput drops.

3. Test with Plain HTTP (If Using HTTPS)
   If your service uses HTTPS, temporarily switch to HTTP and re-run your JMeter tests. If throughput bounces back, the issue is almost certainly related to the updated SSL/TLS stack in u161/u171.

4. Validate Tomcat Connector Configuration
   Check your server.xml connector settings (especially for NIO/NIO2 connectors):

   • Ensure maxConnections and maxThreads are sized appropriately for your server’s CPU/core count. Newer JDKs might handle thread scheduling differently, so adjusting these values could help.
   • Try disabling keep-alive temporarily (keepAliveTimeout="0") to rule out connection pooling overhead changes.

5. Isolate Payload Processing
   Run JMeter tests with minimal POST payloads (e.g., empty or small JSON) vs. large payloads. If the drop only happens with large data, the issue might be in JDK’s serialization/deserialization or buffer handling changes between versions.

Once you’ve identified the root cause, here are targeted fixes:

• Tweak GC Parameters: If GC is the issue, manually set heap/metaspace sizes to match u151’s defaults (or optimize for your workload). For example:

  -Xms2g -Xmx2g -XX:MetaspaceSize=256m -XX:MaxMetaspaceSize=512m
  

• Adjust SSL/TLS Settings: If HTTPS is the problem, force Tomcat to use TLS 1.2 and legacy cipher suites that match u151’s performance. Add these to your HTTPS connector in server.xml:

  sslProtocol="TLSv1.2"
  ciphers="TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384"
  

• Optimize Tomcat Connector: Switch to the APR connector (which uses native libraries for I/O) if you’re using NIO—this can mitigate JDK-level NIO changes. You’ll need to install the Tomcat Native library first.
• Disable Unnecessary Security Checks: If permission checks are adding latency, you can relax the security policy in $JAVA_HOME/jre/lib/security/java.policy (note: only do this in a trusted environment, as it reduces security).

Hope this helps you get your throughput back to where it was. Let me know if you need help interpreting GC logs or connector configs!

内容的提问来源于stack exchange，提问作者Viresh