Java代理服务器：如何从HttpURLConnection读取完整响应流？

问题背景 #

当前使用HttpURLConnection的connection.getInputStream()仅能获取响应体，无法读取包含状态行、响应头在内的完整HTTP响应。需要原封不动转发服务器响应，避免重构响应引发的注入攻击等安全风险。

原有代码 #

public void run()
{
    try
    {
        BufferedReader inFromClient = new BufferedReader(new InputStreamReader(clientSocket.getInputStream()));
        DataOutputStream outForClient = new DataOutputStream(clientSocket.getOutputStream());

        String request = inFromClient.readLine();

        String[] requestParts = request.split(" ");
        String method = requestParts[0]; // Extract request method
        String url = requestParts[1];

        logger.log(Level.INFO, "Outbound {0} request :: {1}", new Object[]{method, url});

        URL destinationUrl = new URL(url);
        HttpURLConnection connection = (HttpURLConnection) destinationUrl.openConnection(Proxy.NO_PROXY);
        connection.setRequestMethod(method); // Set the original request method

        // Forward headers from the client to the destination server
        for (String headerLine = inFromClient.readLine(); headerLine != null && !headerLine.isEmpty(); headerLine = inFromClient.readLine())
        {
            String[] headerParts = headerLine.split(": ", 2);
            connection.setRequestProperty(headerParts[0], headerParts[1]);
        }

        connection.setRequestProperty("Proxy-Status","ADSMSecurity");
        // Send the request body if present
        if (method.equals("POST"))
        {
            connection.setDoOutput(true);
            DataOutputStream outToServer = new DataOutputStream(connection.getOutputStream());
            outToServer.writeBytes("\r\n");
            outToServer.flush();
            outToServer.close();
        }

        connection.connect();
        BufferedReader inputBuffer = new BufferedReader(new InputStreamReader(connection.getInputStream(), "UTF-8"));
        String response = inputBuffer.lines().collect(Collectors.joining("\n"));
        logger.log(Level.INFO,"Response :: {0}",response);
        inFromClient.close();
        outForClient.close();
        clientSocket.close();
    }
    catch (IOException e)
    {
        e.printStackTrace();
    }
}

解决方案 #

一、原生Java实现：直接使用Socket转发原始流 #

HttpURLConnection是封装后的HTTP客户端，会自动解析响应结构，无法直接获取原始响应流。改用底层Socket直接与目标服务器通信，可完整读取并转发原始响应：

import java.io.*;
import java.net.Socket;
import java.net.URL;
import java.nio.charset.StandardCharsets;
import java.util.logging.Level;
import java.util.logging.Logger;

public void run() {
    Logger logger = Logger.getLogger(this.getClass().getName());
    try (Socket clientSocket = this.clientSocket;
         BufferedReader clientReader = new BufferedReader(new InputStreamReader(clientSocket.getInputStream()))) {

        // 读取客户端完整请求内容（状态行、头、体）
        StringBuilder requestBuilder = new StringBuilder();
        String line;
        // 读取状态行和响应头
        while ((line = clientReader.readLine()) != null && !line.isEmpty()) {
            requestBuilder.append(line).append("\r\n");
        }
        requestBuilder.append("\r\n"); // 头与体的分隔符

        // 读取请求体（简化示例，实际需根据Content-Length/Transfer-Encoding处理）
        char[] bodyBuffer = new char[1024];
        int readLen;
        while ((readLen = clientReader.read(bodyBuffer)) != -1) {
            requestBuilder.append(bodyBuffer, 0, readLen);
        }
        String fullRequest = requestBuilder.toString();

        // 解析目标地址
        String firstLine = fullRequest.split("\r\n")[0];
        String urlStr = firstLine.split(" ")[1];
        URL destUrl = new URL(urlStr);
        int destPort = destUrl.getPort() == -1 ? (destUrl.getProtocol().equals("https") ? 443 : 80) : destUrl.getPort();

        try (Socket destSocket = new Socket(destUrl.getHost(), destPort);
             OutputStream destOut = destSocket.getOutputStream();
             InputStream destIn = destSocket.getInputStream();
             OutputStream clientOut = clientSocket.getOutputStream()) {

            // 转发完整请求到目标服务器
            destOut.write(fullRequest.getBytes(StandardCharsets.UTF_8));
            destOut.flush();

            // 直接转发目标服务器的完整响应到客户端
            byte[] buffer = new byte[4096];
            int bytesRead;
            while ((bytesRead = destIn.read(buffer)) != -1) {
                clientOut.write(buffer, 0, bytesRead);
                clientOut.flush();
            }
            logger.log(Level.INFO, "响应已完整转发");
        }

    } catch (IOException e) {
        logger.log(Level.SEVERE, "代理转发失败", e);
    }
}

说明：此方式完全不解析请求/响应结构，直接转发原始字节流，彻底避免重构响应带来的安全风险。HTTPS场景需额外处理SSL握手（如使用SSLSocket）。

二、推荐第三方组件 #

• LittleProxy：基于Netty的轻量级HTTP代理库，专门为代理场景设计，可直接拦截、转发完整HTTP请求与响应，无需手动处理底层流，内置HTTPS代理支持。
• Netty：高性能异步IO框架，通过ByteBuf直接操作原始字节流，完全控制请求/响应转发逻辑，成熟的SSL/TLS支持可轻松实现HTTPS代理。
• Apache HttpClient（底层API）：使用HttpAsyncClient或底层连接管理器，可获取原始响应流，不过需手动拼接响应结构，适合需要自定义逻辑的场景。

示例完整响应结构 #

HTTP/2 400 Bad Request
Vary: Origin
Vary: X-Origin
Vary: Referer
Content-Type: application/json; charset=UTF-8
Date: Sun, 17 Mar 2024 05:44:14 GMT
Server: ESF
Cache-Control: private
Content-Length: 561
X-Xss-Protection: 0
X-Frame-Options: SAMEORIGIN
X-Content-Type-Options: nosniff
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000

Error

内容的提问来源于stack exchange，提问作者test engine