使用Apache HttpClient发起大量本地连接时遇连接拒绝问题求助

我正在为一个接收小型JSON体POST请求的Undertow Web服务器编写测试用例，目标是模拟每秒数百次请求。为降低测试程序自身负载，使用了PoolingHttpClientConnectionManager，测试与服务器在同一机器，目标地址为http://localhost:8080/example。

测试核心代码 #

// http连接池初始化
PoolingHttpClientConnectionManager httpPool = new PoolingHttpClientConnectionManager();
httpPool.setMaxTotal(100);
httpPool.setDefaultMaxPerRoute(100);

// 频繁调用的POST方法
private void post(String url, String json) throws IOException {
    exec.submit(() -> {
        try  {
            CloseableHttpClient client = HttpClients.custom().setConnectionManager(httpPool).build();
            HttpPost post = new HttpPost(url);
            StringEntity entity = new StringEntity(json, ContentType.APPLICATION_JSON);
            post.setEntity(entity);

            try (CloseableHttpResponse response = client.execute(post)) {
                //log(url + ": " + EntityUtils.toString(response.getEntity()));
            }
        } catch (Exception e) {
            e.printStackTrace();
        }
    });
}

异常信息 #

org.apache.hc.client5.http.HttpHostConnectException: Connect to http://localhost:8080 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused: getsockopt
    at java.base/sun.nio.ch.Net.pollConnect(Native Method)
    at java.base/sun.nio.ch.Net.pollConnectNow(Net.java:682)
    at java.base/sun.nio.ch.NioSocketImpl.timedFinishConnect(NioSocketImpl.java:542)
    at java.base/sun.nio.ch.NioSocketImpl.connect(NioSocketImpl.java:592)
    at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:327)
    at java.base/java.net.Socket.connect(Socket.java:751)
    at org.apache.httpcomponents.client5.httpclient5@5.1/org.apache.hc.client5.http.socket.PlainConnectionSocketFactory$1.run(PlainConnectionSocketFactory.java:87)
    at java.base/java.security.AccessController.doPrivileged(AccessController.java:571)
    at org.apache.httpcomponents.client5.httpclient5@5.1/org.apache.hc.client5.http.socket.PlainConnectionSocketFactory.connectSocket(PlainConnectionSocketFactory.java:84)
    at org.apache.httpcomponents.client5.httpclient5@5.1/org.apache.hc.client5.http.impl.io.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:148)
    at org.apache.httpcomponents.client5.httpclient5@5.1/org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:407)
    at org.apache.httpcomponents.client5.httpclient5@5.1/org.apache.hc.client5.http.impl.classic.InternalExecRuntime.connectEndpoint(InternalExecRuntime.java:168)
    at org.apache.httpcomponents.client5.httpclient5@5.1/org.apache.hc.client5.http.impl.classic.InternalExecRuntime.connectEndpoint(InternalExecRuntime.java:178)
    at org.apache.httpcomponents.client5.httpclient5@5.1/org.apache.hc.client5.http.impl.classic.ConnectExec.execute(ConnectExec.java:136)
    at org.apache.httpcomponents.client5.httpclient5@5.1/org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
    at org.apache.httpcomponents.client5.httpclient5@5.1/org.apache.hc.client5.http.impl.classic.ExecChainElement$1.proceed(ExecChainElement.java:57)
    at org.apache.httpcomponents.client5.httpclient5@5.1/org.apache.hc.client5.http.impl.classic.ProtocolExec.execute(ProtocolExec.java:165)
    at org.apache.httpcomponents.client5.httpclient5@5.1/org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
    at org.apache.httpcomponents.client5.httpclient5@5.1/org.apache.hc.client5.http.impl.classic.ExecChainElement$1.proceed(ExecChainElement.java:57)
    at org.apache.httpcomponents.client5.httpclient5@5.1/org.apache.hc.client5.http.impl.classic.HttpRequestRetryExec.execute(HttpRequestRetryExec.java:96)
(后续栈省略)

现象总结 #

• 测试每秒约处理150次请求，运行1分钟后崩溃，所有请求抛出连接拒绝异常
• 服务器无报错，但停止接收新连接
• 立即重启测试直接失败，等待数分钟后重启可正常运行约1分钟、处理近1万次请求后再次崩溃
• 快速重启服务器问题依旧，排除服务器代码问题，怀疑操作系统级资源耗尽（如句柄、端口）

解决思路 #

1. 修复连接池的错误使用（核心问题） #

你的代码每次请求都新建CloseableHttpClient实例，完全浪费了连接池的复用能力。每次新建客户端会导致连接池无法跟踪和复用连接，大量短连接被创建后进入TIME_WAIT状态，快速耗尽本地端口资源。

修正方案：全局只初始化一个CloseableHttpClient实例，复用它处理所有请求：

// 全局唯一的客户端实例
private final CloseableHttpClient client;

// 初始化方法（仅调用一次）
public void initHttpClient() {
    PoolingHttpClientConnectionManager httpPool = new PoolingHttpClientConnectionManager();
    httpPool.setMaxTotal(100);
    httpPool.setDefaultMaxPerRoute(100);
    // 配置连接空闲校验，回收无效连接
    httpPool.setValidateAfterInactivity(5000);
    // 初始化客户端，绑定连接池
    client = HttpClients.custom().setConnectionManager(httpPool).build();
}

private void post(String url, String json) throws IOException {
    exec.submit(() -> {
        try {
            HttpPost post = new HttpPost(url);
            StringEntity entity = new StringEntity(json, ContentType.APPLICATION_JSON);
            post.setEntity(entity);

            try (CloseableHttpResponse response = client.execute(post)) {
                // 必须消费响应实体，避免连接泄漏
                EntityUtils.consume(response.getEntity());
            }
        } catch (Exception e) {
            e.printStackTrace();
        }
    });
}

2. 排查操作系统端口资源 #

连接拒绝+重启需等待的现象，典型是TIME_WAIT端口耗尽：

• Linux查看TIME_WAIT连接数：netstat -ant | grep TIME_WAIT | wc -l
• macOS查看TIME_WAIT连接数：netstat -an | grep TIME_WAIT | wc -l
  如果数值超过万级，说明端口被大量闲置连接占用。

3. 调整TCP参数加速端口回收 #

临时修改系统参数（重启后失效）：

• Linux：

  # 允许复用TIME_WAIT状态的端口
  sysctl -w net.ipv4.tcp_tw_reuse=1
  # 缩短TIME_WAIT超时时间（默认60秒改为15秒）
  sysctl -w net.ipv4.tcp_fin_timeout=15
  

• macOS：

  sudo sysctl -w net.inet.tcp.msl=15000  # 设置为15秒，默认2.5分钟
  sudo sysctl -w net.inet.tcp.reuseaddr=1
  

要永久生效，将参数写入/etc/sysctl.conf（Linux）或/etc/sysctl.conf（macOS）。

4. 优化连接池配置 #

• 根据服务器实际处理能力调整setMaxTotal和DefaultMaxPerRoute，避免超出服务器连接上限
• 配置连接存活时间，自动回收过期连接：

  // 设置连接30秒后过期，自动回收
  PoolingHttpClientConnectionManager httpPool = new PoolingHttpClientConnectionManager(30, TimeUnit.SECONDS);
  

5. 确认Undertow服务器连接上限 #

检查Undertow配置，确保服务器能处理足够并发：

• 调整worker-threads（工作线程数）和max-connections（最大连接数）参数，避免服务器先被打满。

内容的提问来源于stack exchange，提问作者Stik