Java网站TCP连接增多后响应变慢，求ESTABLISHED连接排查处理方案

Hey there! Let's walk through your Java web app connection issues clearly—since you're new to network troubleshooting, I'll break this down into easy-to-follow parts.

ESTABLISHED means a TCP connection is active (either transferring data or waiting for the next request). Here are the most common reasons for a spike:

• Misbehaving clients: Some browsers, crawlers, or custom client apps might not properly send a FIN packet to close the connection after their request is done. This leaves the server hanging onto the ESTABLISHED state indefinitely. You might see this with aggressive scrapers or buggy client code.
• Slow server-side processing: If your Java app takes too long to handle requests (e.g., slow database queries, stuck third-party API calls, unoptimized business logic), the thread handling that request gets blocked. The TCP connection stays ESTABLISHED while waiting for the thread to finish, and new connections stack up as more requests come in.
• Poor thread pool/connection pool config: For example, if your Tomcat server has a maxThreads setting that's too low, it can't keep up with incoming requests. New TCP connections get established (hence ESTABLISHED status) but sit waiting for an available thread to process them. Similarly, misconfigured database or HTTP client pools might leave connections open unnecessarily.
• Overused long connections: If you enabled HTTP Keep-Alive (which is good for performance usually), but the timeout is set too high, idle long connections stay ESTABLISHED for hours, eating up resources.

Let's prioritize fixes based on impact:

• Audit client traffic:
  • Run netstat -anp | grep ESTABLISHED (on Linux) to see which client IPs are holding connections. Check if these are legitimate users or malicious crawlers. For bad actors, use firewall rules or Nginx to limit their request rate.
  • For legitimate clients, work with frontend teams or users to fix any code that fails to close connections properly (e.g., unclosed AJAX requests, missing socket cleanup in client apps).
• Optimize server-side performance:
  • Hunt down slow requests: Use tools like VisualVM or JProfiler to spot blocked threads, or check your Tomcat access logs for requests with long response times. Optimize those bottlenecks—add indexes to slow SQL queries, cache frequent data, or offload heavy tasks to async workers.
  • Tune your app server's thread pool: Adjust Tomcat's maxThreads and acceptCount in server.xml to match your server's CPU/memory. A good starting point is maxThreads = 2 * CPU cores + 1, but tweak based on your traffic.
• Tweak long connection settings:
  • If using Keep-Alive, set a reasonable timeout. In Nginx, add keepalive_timeout 60s; to close idle long connections after a minute. In Tomcat, adjust keepAliveTimeout and connectionTimeout in your connector config.
  • Limit the number of concurrent long connections to prevent resource exhaustion.
• Clean up idle connections:
  • Add heartbeat mechanisms to your app: For custom socket connections, send periodic heartbeats and close connections that don't respond within a timeout.
  • Ensure your app server automatically closes idle connections—most servers (like Tomcat) have built-in timeouts for this, just make sure they're enabled.

This is probably a bigger culprit for your slowdown than ESTABLISHED connections. TIME_WAIT is the state where the server waits after closing a connection to make sure the client gets all packets. Too many of these eat up ports and memory, making it hard to establish new connections. Fixes:

• Adjust Linux kernel parameters (edit /etc/sysctl.conf then run sysctl -p):
  • net.ipv4.tcp_tw_reuse = 1: Allows reusing TIME_WAIT ports for new connections (safe if you enable timestamps below).
  • net.ipv4.tcp_timestamps = 1: Required for tcp_tw_reuse to work securely.
  • net.ipv4.tcp_fin_timeout = 30: Shortens the TIME_WAIT duration from the default 60 seconds to 30.
  • Avoid tcp_tw_recycle: It can cause issues in NAT environments (like cloud networks).

To answer your unfinished question: Don't blindly force-close ESTABLISHED connections—many of them are active requests from users. Focus on fixing the root causes above (slow processing, misbehaving clients, bad config) instead of brute-forcing closures. Only manually close connections if you're certain they're idle and unneeded.

内容的提问来源于stack exchange，提问作者Satish Saini