下载Yahoo Finance股票数据遇JSONDecodeError及403错误求助

问题背景 #

我正在用Python下载纳斯达克Top100股票的股份详情，用于金融分析和股票预测。在将接口返回数据转为Pandas DataFrame时，出现JSON解析错误，根源是所有请求都返回403状态码，但在Chrome浏览器中直接访问相同链接却能正常获取数据。

错误信息 #

JSONDecodeError: Expecting value: line 1 column 1 (char 0)

相关代码 #

shares = []
tickers_done = []
for ticker in tqdm(tickers):
    if ticker in tickers_done:
        continue
    d = requests.get(f"https://query1.finance.yahoo.com/ws/fundamentals-timeseries/v1/finance/timeseries/{ticker}?symbol={ticker}&padTimeSeries=true&type=annualPreferredSharesNumber,annualOrdinarySharesNumber&merge=false&period1=0&period2=2013490868")
    if not d.ok:
      time.sleep(300)
      d = requests.get(f"https://query1.finance.yahoo.com/ws/fundamentals-timeseries/v1/finance/timeseries/{ticker}?symbol={ticker}&padTimeSeries=true&type=annualPreferredSharesNumber,annualOrdinarySharesNumber&merge=false&period1=0&period2=2013490868")
    ctn = d.json()['timeseries']['result']
    dct = dict()
    for n in ctn:
        type = n['meta']['type'][0]
        dct[type] = dict()
        if type in n:
            for o in n[type]:
                if o is not None:
                    dct[type][o['asOfDate']] = o['reportedValue']['raw']
    df = pd.DataFrame.from_dict(dct)
    df['symbol'] = ticker
    shares.append(df)
    tickers_done.append(ticker)
    time.sleep(1)
# save dataframe
df = pd.concat(shares)
df['date'] = df.index
df.to_csv(f"data/{project}/shares.csv", index=False)

排查情况 #

• 检查所有ticker请求的状态码，结果全部为403，检查代码如下：

for x in tickers:
    d = requests.get(f"https://query1.finance.yahoo.com/ws/fundamentals-timeseries/v1/finance/timeseries/{x}?symbol={x}&padTimeSeries=true&type=annualPreferredSharesNumber,annualOrdinarySharesNumber&merge=false&period1=0&period2=2013490868")
    print(d.status_code)

• 直接在Chrome浏览器访问单支股票（如AAPL）的相同请求链接，可正常获取JSON格式的股份数据。

解决方案 #

Yahoo Finance接口会校验请求头信息，尤其是User-Agent字段，模拟浏览器请求即可解决403问题，同时优化错误处理避免崩溃：

1. 添加请求头模拟浏览器 #

修改请求代码，加入浏览器的User-Agent头：

headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36'
}
# 替换原请求代码
d = requests.get(f"https://query1.finance.yahoo.com/ws/fundamentals-timeseries/v1/finance/timeseries/{ticker}?symbol={ticker}&padTimeSeries=true&type=annualPreferredSharesNumber,annualOrdinarySharesNumber&merge=false&period1=0&period2=2013490868", headers=headers)

2. 优化错误处理逻辑 #

即使重试后仍返回非200状态，跳过当前ticker避免后续解析报错：

if not d.ok:
    print(f"请求{ticker}失败，状态码：{d.status_code}")
    continue

3. 使用第三方封装库更稳定 #

推荐使用yfinance库，它已经处理了请求头和接口适配问题，代码更简洁：

import yfinance as yf

ticker = yf.Ticker("AAPL")
# 获取完整年度股份数据
shares_data = ticker.get_shares_full(start='1900-01-01', end='2023-12-31')

内容的提问来源于stack exchange，提问作者NeuralAI