Spring Boot RestClient请求api.nasdaq.com时挂起，求排查方向

我的Spring Boot RestClient请求https://google.com正常，但请求https://api.nasdaq.com时始终挂起，相关代码如下：

RestClient restclient = RestClient.builder()
                                    .baseUrl("https://api.nasdaq.com")
                                    .defaultHeader(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON_VALUE)
                                    .build(); 

ResponseEntity<String> resp = restclient.get().uri("/api/quote/TSLA/historical?assetclass=stocks&fromdate=2024-04-01&limit=9999&todate=2024-04-08").retrieve().toEntity(String.class); 
System.out.println(resp);

查阅大量资料后仍未解决，请求挂起时的日志信息：

2024-04-07T16:01:27.261-04:00 DEBUG 2986853 --- [dto] [   scheduling-1] o.a.h.c.http.protocol.RequestAddCookies  : ex-0000000001 Cookie spec selected: strict
2024-04-07T16:01:27.275-04:00 DEBUG 2986853 --- [dto] [   scheduling-1] o.a.h.c.h.i.classic.InternalHttpClient   : ep-0000000001 start execution ex-0000000001
2024-04-07T16:01:27.275-04:00 DEBUG 2986853 --- [dto] [   scheduling-1] h.i.i.PoolingHttpClientConnectionManager : ep-0000000001 executing exchange ex-0000000001 over http-outgoing-0
2024-04-07T16:01:27.277-04:00 DEBUG 2986853 --- [dto] [   scheduling-1] org.apache.hc.client5.http.headers       : http-outgoing-0 >> GET /api/quote/TSLA/historical?assetclass=stocks&fromdate=2024-04-01&limit=9999&todate=2024-04-08 HTTP/1.1
2024-04-07T16:01:27.277-04:00 DEBUG 2986853 --- [dto] [   scheduling-1] org.apache.hc.client5.http.headers       : http-outgoing-0 >> Content-Type: application/json
2024-04-07T16:01:27.277-04:00 DEBUG 2986853 --- [dto] [   scheduling-1] org.apache.hc.client5.http.headers       : http-outgoing-0 >> Accept-Encoding: gzip, x-gzip, deflate
2024-04-07T16:01:27.277-04:00 DEBUG 2986853 --- [dto] [   scheduling-1] org.apache.hc.client5.http.headers       : http-outgoing-0 >> Host: api.nasdaq.com
2024-04-07T16:01:27.277-04:00 DEBUG 2986853 --- [dto] [   scheduling-1] org.apache.hc.client5.http.headers       : http-outgoing-0 >> Connection: keep-alive
2024-04-07T16:01:27.277-04:00 DEBUG 2986853 --- [dto] [   scheduling-1] org.apache.hc.client5.http.headers       : http-outgoing-0 >> User-Agent: Apache-HttpClient/5.2.3 (Java/17.0.10)
2024-04-07T16:01:27.278-04:00 DEBUG 2986853 --- [dto] [   scheduling-1] org.apache.hc.client5.http.wire          : http-outgoing-0 >> "GET /api/quote/TSLA/historical?assetclass=stocks&fromdate=2024-04-01&limit=9999&todate=2024-04-08 HTTP/1.1[
][
]"
2024-04-07T16:01:27.278-04:00 DEBUG 2986853 --- [dto] [   scheduling-1] org.apache.hc.client5.http.wire          : http-outgoing-0 >> "Content-Type: application/json[
][
]"
2024-04-07T16:01:27.278-04:00 DEBUG 2986853 --- [dto] [   scheduling-1] org.apache.hc.client5.http.wire          : http-outgoing-0 >> "Accept-Encoding: gzip, x-gzip, deflate[
][
]"
2024-04-07T16:01:27.278-04:00 DEBUG 2986853 --- [dto] [   scheduling-1] org.apache.hc.client5.http.wire          : http-outgoing-0 >> "Host: api.nasdaq.com[
][
]"
2024-04-07T16:01:27.278-04:00 DEBUG 2986853 --- [dto] [   scheduling-1] org.apache.hc.client5.http.wire          : http-outgoing-0 >> "Connection: keep-alive[
][
]"
2024-04-07T16:01:27.278-04:00 DEBUG 2986853 --- [dto] [   scheduling-1] org.apache.hc.client5.http.wire          : http-outgoing-0 >> "User-Agent: Apache-HttpClient/5.2.3 (Java/17.0.10)[
][
]"
2024-04-07T16:01:27.278-04:00 DEBUG 2986853 --- [dto] [   scheduling-1] org.apache.hc.client5.http.wire          : http-outgoing-0 >> "[
][
]"
2024-04-07T16:01:27.588-04:00 DEBUG 2986853 --- [dto] [nio-9400-Poller] org.apache.tomcat.util.net.NioEndpoint   : timeout completed: keys processed=0; now=1712520087588; nextExpiration=1712520087576; keyCount=0; hasEvents=false; eval=false

可能的问题及解决方案 #

• User-Agent被服务器拦截：日志显示当前请求的User-Agent为Apache-HttpClient/5.2.3 (Java/17.0.10)，这类标识容易被公开API的反爬机制拦截。建议替换为浏览器常用的User-Agent，比如Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36。
• GET请求错误设置Content-Type头：当前代码给GET请求设置了Content-Type: application/json，GET请求通常不需要该头，这可能被服务器判定为异常请求，建议移除这个默认头。
• 未配置请求超时：代码中没有设置连接和读取超时，导致请求挂起后无限等待。建议添加超时配置，避免长时间阻塞。
• 缺少必要Cookie验证：api.nasdaq.com可能需要特定会话Cookie才能正常响应，可先通过浏览器访问该接口，复制请求中的Cookie信息添加到请求头中。

修改后的示例代码 #

import org.springframework.http.HttpHeaders;
import org.springframework.http.client.ClientHttpRequestFactory;
import org.springframework.http.client.HttpComponentsClientHttpRequestFactory;
import org.springframework.web.client.RestClient;

import java.time.Duration;

public class NasdaqClient {
    public static void main(String[] args) {
        RestClient restclient = RestClient.builder()
                .baseUrl("https://api.nasdaq.com")
                .defaultHeader(HttpHeaders.USER_AGENT, "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36")
                // 可根据浏览器请求添加必要Cookie
                // .defaultHeader(HttpHeaders.COOKIE, "your-cookie-value")
                .requestFactory(clientHttpRequestFactory())
                .build();

        ResponseEntity<String> resp = restclient.get()
                .uri("/api/quote/TSLA/historical?assetclass=stocks&fromdate=2024-04-01&limit=9999&todate=2024-04-08")
                .retrieve()
                .toEntity(String.class);
        System.out.println(resp);
    }

    private static ClientHttpRequestFactory clientHttpRequestFactory() {
        HttpComponentsClientHttpRequestFactory factory = new HttpComponentsClientHttpRequestFactory();
        factory.setConnectTimeout(Duration.ofSeconds(10));
        factory.setReadTimeout(Duration.ofSeconds(10));
        return factory;
    }
}

内容的提问来源于stack exchange，提问作者PLy