Python Requests与cURL请求NSE印度网站无响应的解决问询

我尝试用Python的requests库爬取NSE印度网站上某特定公司的页面，目标是获取HTML内容用Beautiful Soup 4解析，初始代码如下：

url = 'https://www.nseindia.com/get-quotes/equity?symbol=20MICRONS'

response = requests.get(url)

但请求发起后数分钟都没有响应，甚至导致系统崩溃。

参考相关问题建议，我尝试添加User-Agent请求头，代码如下：

url = 'https://www.nseindia.com/get-quotes/equity?symbol=20MICRONS'
headers = {
    "User-Agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36",
}

response = requests.get(url, headers=headers)

这个方法没有效果，排除本地性能问题后，我尝试切换到IPv4（IPv6无法解析该域名），但用curl以IPv4+HTTP/2请求时报错：

$ curl --ipv4 'https://www.nseindia.com/get-quotes/equity?symbol=20MICRONS' # -v removed to focus on the error
curl: (92) HTTP/2 stream 1 was not closed cleanly: INTERNAL_ERROR (err 2)

改用HTTP/1.1后，连接成功建立，但请求始终无法完成，curl输出如下：

curl --ipv4 -v 'https://www.nseindia.com/get-quotes/equity?symbol=20MICRONS' --http1.1
* Host www.nseindia.com:443 was resolved.
* IPv6: (none)
* IPv4: 184.29.25.143
*   Trying 184.29.25.143:443...
* Connected to www.nseindia.com (184.29.25.143) port 443
* ALPN: curl offers http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: none
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384 / x25519 / RSASSA-PSS
* ALPN: server accepted http/1.1
* Server certificate:
*  subject: C=IN; ST=Maharashtra; L=Mumbai; O=National Stock Exchange of India Ltd.; CN=www.nseindia.com
*  start date: May 28 00:00:00 2024 GMT
*  expire date: May 22 23:59:59 2025 GMT
*  subjectAltName: host "www.nseindia.com" matched cert's "www.nseindia.com"
*  issuer: C=US; O=DigiCert Inc; OU=www.digicert.com; CN=GeoTrust RSA CA 2018
*  SSL certificate verify ok.
*   Certificate level 0: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
*   Certificate level 1: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
*   Certificate level 2: Public key type RSA (2048/112 Bits/secBits), signed using sha1WithRSAEncryption
* using HTTP/1.x
> GET /get-quotes/equity?symbol=20MICRONS HTTP/1.1
> Host: www.nseindia.com
> User-Agent: curl/8.8.0
> Accept: */*
> 
* Request completely sent off
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing

之后我参考NSE相关问题的解决方案，尝试用Session携带cookies，代码如下：

baseurl = 'https://www.nseindia.com/'
url = 'https://www.nseindia.com/get-quotes/equity?symbol=20MICRONS'
headers = {
    'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36',
    'accept-language': 'en,gu;q=0.9,hi;q=0.8',
    'accept-encoding': 'gzip, deflate, br'
}

session = requests.Session()
request = session.get(baseurl, headers=headers, timeout=5)
cookies = dict(request.cookies)
response = session.get(url, headers=headers, timeout=5, cookies=cookies)

这个方案依旧没有效果，添加Accept头后问题也没有改善。

请问该如何推进排查？这个问题是请求过慢导致的，还是存在某种请求错误？

内容的提问来源于stack exchange，提问作者Shirsak