Python3.9调用NSE REST API遇Gateway Timeout问题求解

我用Python 3.9查询NSE印度证券交易所（目标股票落地页：https://www.nseindia.com/get-quotes/equity?symbol=EQUITASBNK）的历史价量数据时，GET请求出现Gateway Timeout。当前逻辑是：

1. 请求目标股票的落地页，返回200状态码
2. 提取响应中的Cookie
3. 携带Cookie和参数请求历史数据，但这一步请求一直挂起无响应

原代码 #

class NSE(Exchange):
    def __init__(self):
        self.url_landing = "https://www.nseindia.com/get-quotes/equity?"
        self.url_quotes ="https://www.nseindia.com/api/historical/cm/equity?"

    def fetchbulkprices(self, ticker, fromdate, todate):
          sys.stderr.write("Querying Ticker = {} fromdate = {} todate {} \n".format(ticker, fromdate, todate))
          headers = {
            "authority": "www.nseindia.com",
            "method": "GET",
            "path": "/api/historical/cm/equity?symbol=" + ticker + "&series = [%22EQ%22]&from=" + fromdate + "&to="+ todate+ "&csv=true",
           "scheme": "https",
           "accept": "*/*",
           "accept-Encoding": "gzip, deflate, br",
           "accept-Language": "en-GB,en-US;q=0.9,en;q=0.8",
           "referer": "https://www.nseindia.com/get-quotes/equity?symbol="+ticker,
           "sec-ch-ua": "Google Chrome" + ";" + "v=""111"", ""Not(A:Brand""" + ";" + "v=""8""" + ",""Chromium""",
           "sec-ch-ua-mobile" : "?0",
           "sec-ch-ua-platform" : "Windows",
           "sec-fetch-dest": "empty",
           "sec-fetch-mode": "cors",
           "sec-fetch-site": "same-origin",
           "user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.93 Safari/537.36",
           "x -requested-with": "XMLHttpRequest"
           }
         session = requests.Session()
         params = {"symbol": ticker}
         response = requests.get(self.url_landing, params=params, headers=headers)
        cookies = response.cookies
        params = {"symbol": ticker, "series": "[%22EQ%22]", "fromDate": fromdate,"toDate": todate, "csv": True}
        response = session.get(self.url_quotes, params=params, headers=headers, cookies=cookies)
        if response.status_code == 200:
          sys.stderr.write("Queried successfully")

示例查询参数 #

• AAVAS, 18-09-2020, 23-01-2021
• EQUITASBNK, 18-09-2020, 23-01-2021
• MASTEK, 18-09-2020, 23-01-2021

问题排查与修正方案 #

1. 清理错误的Headers配置

• 移除method、path、scheme、authority等requests会自动处理的字段，手动添加会干扰请求逻辑
• 修正格式错误的sec-ch-ua字段，直接使用标准浏览器标识即可
• 修复x -requested-with的空格问题，改为X-Requested-With
• 只保留必要的Headers字段，减少冗余信息

2. 正确使用Session管理Cookie

• 用Session对象发起所有请求，Session会自动保存和携带Cookie，无需手动提取传递
• 避免混用requests.get和session.get，导致Cookie无法正确复用

3. 修正参数格式

• series参数无需手动URL编码，直接传["EQ"]即可，requests会自动处理编码
• csv参数改为字符串格式"true"，匹配API要求

4. 添加超时限制

• 给所有请求添加timeout参数，避免请求无限挂起

修正后的代码 #

import sys
import requests

class NSE:
    def __init__(self):
        self.url_landing = "https://www.nseindia.com/get-quotes/equity"
        self.url_quotes = "https://www.nseindia.com/api/historical/cm/equity"

    def fetchbulkprices(self, ticker, fromdate, todate):
        sys.stderr.write(f"Querying Ticker = {ticker} fromdate = {fromdate} todate {todate}\n")
        
        headers = {
            "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/111.0.0.0 Safari/537.36",
            "Accept": "*/*",
            "Accept-Language": "en-GB,en-US;q=0.9,en;q=0.8",
            "Referer": f"https://www.nseindia.com/get-quotes/equity?symbol={ticker}",
            "X-Requested-With": "XMLHttpRequest"
        }

        session = requests.Session()
        # 用Session请求落地页，自动保存Cookie
        try:
            response = session.get(self.url_landing, params={"symbol": ticker}, headers=headers, timeout=10)
            response.raise_for_status()
        except requests.exceptions.RequestException as e:
            sys.stderr.write(f"Landing page request failed: {str(e)}\n")
            return None

        # 构造历史数据请求参数
        params = {
            "symbol": ticker,
            "series": '["EQ"]',
            "fromDate": fromdate,
            "toDate": todate,
            "csv": "true"
        }
        
        try:
            # 用Session请求历史数据，自动携带Cookie
            response = session.get(self.url_quotes, params=params, headers=headers, timeout=10)
            if response.status_code == 200:
                sys.stderr.write("Queried successfully\n")
                return response.text
            else:
                sys.stderr.write(f"Request failed with status code: {response.status_code}\n")
                return None
        except requests.exceptions.RequestException as e:
            sys.stderr.write(f"Historical data request failed: {str(e)}\n")
            return None

额外注意事项 #

• NSE有反爬机制，请求间隔不要过短，建议添加time.sleep(2)控制频率
• 定期更新User-Agent，避免被识别为爬虫
• 如果仍出现超时，可尝试更换IP或使用代理（需遵守平台规则）

内容的提问来源于stack exchange，提问作者Soham