特定PDF链接curl HTTP/2调用失败，同类链接正常的技术问题

问题概况 #

使用curl下载PDF时，BSE的链接可正常工作，但NSE的PDF链接触发HTTP/2 stream 1 was not closed cleanly: INTERNAL_ERROR (err 2)错误；降级到HTTP/1.1后下载启动但无法终止，而Chrome能正常渲染两个PDF文件。

可正常执行的命令 #

curl -v https://www.bseindia.com/bseplus/AnnualReport/543258/74183543258.pdf -o bse_file.pdf

执行失败的命令 #

curl -v https://nsearchives.nseindia.com/content/equities/IPO_RHP_UNICOMM.pdf -o nse_file.pdf

错误信息 #

curl: (92) HTTP/2 stream 1 was not closed cleanly: INTERNAL_ERROR (err 2)

curl版本信息 #

curl --version
curl 8.4.0 (x86_64-apple-darwin23.0) libcurl/8.4.0 (SecureTransport) LibreSSL/3.3.6 zlib/1.2.12 nghttp2/1.58.0

错误调用详细日志 #

$ curl -v https://nsearchives.nseindia.com/content/equities/IPO_RHP_UNICOMM.pdf -o nse_file.pdf
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying 23.46.207.147:443...
* Connected to nsearchives.nseindia.com (23.46.207.147) port 443
* ALPN: curl offers h2,http/1.1
* (304) (OUT), TLS handshake, Client hello (1):
} [329 bytes data]
*  CAfile: /etc/ssl/cert.pem
*  CApath: none
* (304) (IN), TLS handshake, Server hello (2):
{ [122 bytes data]
* (304) (IN), TLS handshake, Unknown (8):
{ [29 bytes data]
* (304) (IN), TLS handshake, Certificate (11):
{ [4274 bytes data]
* (304) (IN), TLS handshake, CERT verify (15):
{ [80 bytes data]
* (304) (IN), TLS handshake, Finished (20):
{ [36 bytes data]
* (304) (OUT), TLS handshake, Finished (20):
} [36 bytes data]
* SSL connection using TLSv1.3 / AEAD-CHACHA20-POLY1305-SHA256
* ALPN: server accepted h2
* Server certificate:
*  subject: C=IN; ST=Maharashtra; L=Mumbai; O=National Stock Exchange of India Ltd; CN=www.nseindia.com
*  start date: Aug 21 00:00:00 2024 GMT
*  expire date: Jan 28 23:59:59 2025 GMT
*  subjectAltName: host "nsearchives.nseindia.com" matched cert's "nsearchives.nseindia.com"
*  issuer: C=US; O=DigiCert Inc; CN=DigiCert TLS RSA SHA256 2020 CA1
*  SSL certificate verify ok.
* using HTTP/2
* [HTTP/2] [1] OPENED stream for https://nsearchives.nseindia.com/content/equities/IPO_RHP_UNICOMM.pdf
* [HTTP/2] [1] [:method: GET]
* [HTTP/2] [1] [:scheme: https]
* [HTTP/2] [1] [:authority: nsearchives.nseindia.com]
* [HTTP/2] [1] [:path: /content/equities/IPO_RHP_UNICOMM.pdf]
* [HTTP/2] [1] [user-agent: curl/8.4.0]
* [HTTP/2] [1] [accept: */*]
> GET /content/equities/IPO_RHP_UNICOMM.pdf HTTP/2
> Host: nsearchives.nseindia.com
> User-Agent: curl/8.4.0
> Accept: */*
>
* HTTP/2 stream 1 was not closed cleanly: INTERNAL_ERROR (err 2)
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
* Connection #0 to host nsearchives.nseindia.com left intact
curl: (92) HTTP/2 stream 1 was not closed cleanly: INTERNAL_ERROR (err 2)

已尝试方案 #

使用--http1.1参数降级协议后，下载启动但无法终止：

curl -v --http1.1 https://nsearchives.nseindia.com/content/equities/IPO_RHP_UNICOMM.pdf -o curl_file.pdf

解决方案 #

1. 模拟浏览器User-Agent并添加超时限制 #

服务器可能对curl默认的User-Agent做了拦截，模拟Chrome的UA并配合HTTP/1.1，同时设置超时避免无限等待：

curl -v --http1.1 -A "Mozilla/5.0 (Macintosh; Intel Mac OS X 14_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" --max-time 60 https://nsearchives.nseindia.com/content/equities/IPO_RHP_UNICOMM.pdf -o nse_file.pdf

2. 禁用HTTP/2并关闭长连接 #

直接禁用HTTP/2协议，同时关闭长连接避免卡住：

curl -v --no-http2 --no-keepalive https://nsearchives.nseindia.com/content/equities/IPO_RHP_UNICOMM.pdf -o nse_file.pdf

3. 携带会话Cookie（从Chrome导出） #

若服务器需要验证会话，可从Chrome中导出对应网站的Cookie，通过--cookie参数传入：

curl -v --http1.1 -A "Mozilla/5.0 (Macintosh; Intel Mac OS X 14_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" --cookie "your_cookie_string_here" https://nsearchives.nseindia.com/content/equities/IPO_RHP_UNICOMM.pdf -o nse_file.pdf

内容的提问来源于stack exchange，提问作者Nitiraj