Apache HttpClient重定向URL被自动解码，如何解决？

使用Apache HttpClient 4.5.14发起请求时，收到302重定向响应，响应头Location包含URL编码的地址（例如%27单引号的编码），但HttpClient会自动解码该URL并请求解码后的地址，导致请求不符合预期。

现象日志 #

响应返回的Location头：

2023-10-31 07:25:38.899 DEBUG --- [ main] org.apache.http.headers : http-outgoing-2 << Location: https://[hostname-redacted]/temp/AB-C0244_BONNIE_BANKS_O_LOCH_LOMOND16987335293607/AB-C0244_TK22.2_BONNIE_BANKS_O%27_LOCH_LOMOND_General_Underscore_Abaco_Music_Library_(PRS).wav?Expires=1698733651&Signature=ZkxxNJhbLrX84ykFuFA7pKEhc05pCrUGJ8oQNjUbCswvh5o8K2k7W3HlG16zrX4EO2co4M1074X2uoHAlpTjnqhSz6FSnWeFud0LIEU6z99nvYmll6bgj1VQg~FqFz2aGLHF9qzRIeM2e73gWBU8bu5EVg5X5n-m8gdyz6-zg5M_&Key-Pair-Id=APKAIBH6A7AA3GUADXUQ

HttpClient实际请求的URL：

2023-10-31 07:25:38.902 DEBUG --- [ main] o.a.http.impl.execchain.MainClientExec : Executing request GET /temp/AB-C0244_BONNIE_BANKS_O_LOCH_LOMOND16987335293607/AB-C0244_TK22.2_BONNIE_BANKS_O'LOCH_LOMOND_General_Underscore_Abaco_Music_Library(PRS).wav?Expires=1698733651&Signature=ZkxxNJhbLrX84ykFuFA7pKEhc05pCrUGJ8oQNjUbCswvh5o8K2k7W3HlG16zrX4EO2co4M1074X2uoHAlpTjnqhSz6FSnWeFud0LIEU6z99nvYmll6bgj1VQg~FqFz2aGLHF9qzRIeM2e73gWBU8bu5EVg5X5n-m8gdyz6-zg5M_&Key-Pair-Id=APKAIBH6A7AA3GUADXUQ HTTP/1.1

问题根源 #

自定义RedirectStrategy无效，因为URL解码发生在RedirectStrategy传递URI之后。进一步排查发现，DefaultRedirectStrategy的getLocationURI方法中，当config.isNormalizeUri()为true时，会执行URIUtils.normalizeSyntax(uri)。按照URI标准化定义，该操作仅应解码非保留字符的百分编码，但实际却解码了%27（单引号属于保留字符），不符合预期。

复现代码 #

依赖配置（pom.xml） #

<dependency>
    <groupId>org.apache.httpcomponents</groupId>
    <artifactId>httpclient</artifactId>
    <version>4.5.14</version>
</dependency>

测试代码 #

开启org.apache.http调试日志后，运行以下代码可复现问题：当重定向到/test%27in时，HttpClient会请求/test'in：

package test;

import com.sun.net.httpserver.HttpExchange;
import com.sun.net.httpserver.HttpHandler;
import com.sun.net.httpserver.HttpServer;
import java.io.IOException;
import java.io.OutputStream;
import java.net.InetSocketAddress;
import org.apache.http.client.config.RequestConfig;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;

public class ApacheTest {

    public static void main(String[] args) throws IOException, InterruptedException {
        HttpServer server = HttpServer.create(new InetSocketAddress(8000), 0);
        server.createContext("/test", new RedirectHttpHandler());
        server.createContext("/test'in", new SuccessHttpHandler());
        server.setExecutor(null);
        server.start();

        String uri = "http://127.0.0.1:8000/test";
        ApacheTest t = new ApacheTest();
        t.doRequest(uri);

        server.stop(0);
    }
    
    private CloseableHttpClient getClient() {
        RequestConfig requestConfig =
                RequestConfig.custom().setNormalizeUri(true).build();

        CloseableHttpClient httpClient =
                HttpClients.custom().setDefaultRequestConfig(requestConfig).build();
        
        return httpClient;
    }
    
    private void doRequest(String uri) {
        try {
            CloseableHttpClient c = getClient();
            HttpGet get = new HttpGet(uri);
            CloseableHttpResponse response = c.execute(get);
            if (response.getStatusLine().getStatusCode() != 200) {
                throw new Exception("Expected 200 OK");
            }

            System.out.println("Got 200 OK!");

        } catch (Exception e) {
            e.printStackTrace();
        }
    }
    
    public static class RedirectHttpHandler implements HttpHandler {
        @Override
        public void handle(HttpExchange t) throws IOException {
            t.getResponseHeaders().add("Location", "/test%27in");
            t.sendResponseHeaders(302, 0);
            OutputStream os = t.getResponseBody();
            os.close();
        }
    }    
    
    public static class SuccessHttpHandler implements HttpHandler {
        @Override
        public void handle(HttpExchange t) throws IOException {
            String response = "This is the response";
            t.sendResponseHeaders(200, response.length());
            OutputStream os = t.getResponseBody();
            os.write(response.getBytes());
            os.close();
        }
    }        
}

针对该问题，有三种可行处理方式：

1. 关闭URI标准化 #

直接在RequestConfig中关闭URI标准化功能，这样URIUtils.normalizeSyntax不会执行，Location头的编码会被保留：

private CloseableHttpClient getClient() {
    RequestConfig requestConfig =
            RequestConfig.custom()
                    .setNormalizeUri(false) // 关闭URI标准化
                    .build();

    CloseableHttpClient httpClient =
            HttpClients.custom().setDefaultRequestConfig(requestConfig).build();
    
    return httpClient;
}

注意：关闭标准化后，HttpClient不会自动处理URI中的冗余路径（如../），需自行确保Location头的URI格式正确。

2. 自定义RedirectStrategy并重写getLocationURI #

如果需要保留部分标准化功能，同时避免解码保留字符，可以自定义RedirectStrategy，重写getLocationURI方法，跳过URIUtils.normalizeSyntax调用，或替换为符合预期的标准化逻辑：

import org.apache.http.client.RedirectStrategy;
import org.apache.http.client.config.RequestConfig;
import org.apache.http.client.protocol.HttpClientContext;
import org.apache.http.impl.client.DefaultRedirectStrategy;
import org.apache.http.protocol.HttpContext;
import org.apache.http.HttpResponse;
import java.net.URI;

public class NoNormalizeRedirectStrategy extends DefaultRedirectStrategy {
    @Override
    protected URI getLocationURI(HttpResponse response, HttpContext context) throws org.apache.http.ProtocolException {
        HttpClientContext clientContext = HttpClientContext.adapt(context);
        RequestConfig config = clientContext.getRequestConfig();
        
        // 临时关闭标准化
        RequestConfig tempConfig = RequestConfig.copy(config)
                .setNormalizeUri(false)
                .build();
        clientContext.setRequestConfig(tempConfig);
        
        URI uri = super.getLocationURI(response, context);
        
        // 恢复原始配置
        clientContext.setRequestConfig(config);
        return uri;
    }
}

创建HttpClient时使用该自定义策略：

private CloseableHttpClient getClient() {
    RequestConfig requestConfig =
            RequestConfig.custom().setNormalizeUri(true).build();

    CloseableHttpClient httpClient =
            HttpClients.custom()
                    .setDefaultRequestConfig(requestConfig)
                    .setRedirectStrategy(new NoNormalizeRedirectStrategy()) // 应用自定义策略
                    .build();
    
    return httpClient;
}

3. 升级HttpClient版本 #

该问题在HttpClient后续版本（如5.x系列）中已被修复，若项目允许，升级到最新稳定版可直接解决URI标准化不符合预期的问题。

内容的提问来源于stack exchange，提问作者sbrattla