如何用Python获取Yahoo Finance加密后的行情数据？

近期Yahoo对行情数据进行了加密处理，导致通过Python脚本直接请求时无法获取明文数据，但在网页浏览器中却能正常显示未加密的行情。以下是遇到问题的代码：

import requests
from bs4 import BeautifulSoup
import re
import json
import pandas as pd
from datetime import datetime as dt
import time

def _get_headers():
    return {"accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8",
            "authority":"ca.finance.yahoo.com",
            "accept-encoding": "gzip, deflate, br",
            "accept-language": "en;q=0.9",
            "cache-control": "no-cache",
            "dnt": "1",
            "sec-ch-ua-platform": "Windows",
            "sec-fetch-dest": "document",
            "sec-fetch-mode": "navigate",
            "sec-fetch-user": "?1",
            "upgrade-insecure-requests": "1",
            "user-agent": "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.0.0 Safari/537.36"}

def get_yahoo_finance_price(ticker):
    time.sleep(5)
    url = 'https://finance.yahoo.com/quote/'+ticker+'/history?p='+ticker

    html = requests.get(url,  headers=_get_headers(), timeout=(3.05, 21)).text

    soup = BeautifulSoup(html,'html.parser')
    soup_script = soup.find("script",text=re.compile("root.App.main")).text
    matched = re.search("root.App.main\s+=\s+(\{.*\})",soup_script)
    # if matched:
    json_script = json.loads(matched.group(1))
    print(json_script)
    data = json_script['context']['dispatcher']['stores']['HistoricalPriceStore']['prices'][0]
    df = pd.DataFrame({'date': dt.fromtimestamp(data['date']).strftime("%Y-%m-%d"),
                     'close': round(data['close'], 2),
                     "adjusted close": round(data['adjclose'], 2),
                     'volume': data['volume'],
                     'open': round(data['open'], 2),
                     'high': round(data['high'], 2),
                     'low': round(data['low'], 2),
                     }, index=[0])
    return df

问题原因 #

Yahoo Finance现在会对无有效会话的请求返回加密或混淆的数据，而浏览器访问时会自动建立会话并获取验证Cookie，因此能正常解析明文数据。之前的代码仅设置了请求头，未处理会话Cookie，导致无法获取正确的未加密内容。

解决方案 #

方案1：使用Session保持会话并获取Cookie #

通过requests.Session()模拟浏览器的会话流程，先访问Yahoo Finance首页获取必要Cookie，再请求目标行情页面：

import requests
from bs4 import BeautifulSoup
import re
import json
import pandas as pd
from datetime import datetime as dt
import time

def _get_headers():
    return {"accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8",
            "accept-language": "en;q=0.9",
            "cache-control": "no-cache",
            "dnt": "1",
            "sec-ch-ua-platform": "Windows",
            "sec-fetch-dest": "document",
            "sec-fetch-mode": "navigate",
            "sec-fetch-user": "?1",
            "upgrade-insecure-requests": "1",
            "user-agent": "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.0.0 Safari/537.36"}

def get_yahoo_finance_price(ticker):
    time.sleep(5)
    session = requests.Session()
    # 先访问首页获取Cookie
    session.get("https://finance.yahoo.com/", headers=_get_headers(), timeout=(3.05, 21))
    url = f'https://finance.yahoo.com/quote/{ticker}/history?p={ticker}'
    response = session.get(url, headers=_get_headers(), timeout=(3.05, 21))
    response.encoding = 'utf-8'
    html = response.text

    soup = BeautifulSoup(html,'html.parser')
    soup_script = soup.find("script", text=re.compile("root.App.main"))
    if not soup_script:
        raise Exception("无法找到目标脚本数据")
    matched = re.search(r"root.App.main\s+=\s+(\{.*\});", soup_script.text)
    if not matched:
        raise Exception("无法匹配目标JSON数据")
    json_script = json.loads(matched.group(1))
    prices = json_script['context']['dispatcher']['stores']['HistoricalPriceStore']['prices']
    if not prices:
        raise Exception("未获取到行情数据")
    data = prices[0]
    df = pd.DataFrame({
        'date': dt.fromtimestamp(data['date']).strftime("%Y-%m-%d"),
        'close': round(data['close'], 2),
        "adjusted close": round(data['adjclose'], 2),
        'volume': data['volume'],
        'open': round(data['open'], 2),
        'high': round(data['high'], 2),
        'low': round(data['low'], 2),
    }, index=[0])
    return df

方案2：使用专门的Yahoo Finance库（推荐） #

使用yfinance库，它已经适配了Yahoo的最新反爬和加密机制，无需手动处理请求细节：

1. 安装库：

pip install yfinance

2. 使用示例：

import yfinance as yf
import pandas as pd

def get_yahoo_finance_price(ticker):
    stock = yf.Ticker(ticker)
    # 获取最新一条历史数据（可调整period参数获取更多）
    hist = stock.history(period='1d')
    if hist.empty:
        raise Exception("未获取到行情数据")
    latest_data = hist.iloc[-1]
    df = pd.DataFrame({
        'date': latest_data.name.strftime("%Y-%m-%d"),
        'close': round(latest_data['Close'], 2),
        "adjusted close": round(latest_data['Close'], 2),  # yfinance中Close已包含复权
        'volume': latest_data['Volume'],
        'open': round(latest_data['Open'], 2),
        'high': round(latest_data['High'], 2),
        'low': round(latest_data['Low'], 2),
    }, index=[0])
    return df

注意事项 #

• 方案1中需注意会话的有效性，若频繁请求可能仍会被限制，可适当增加请求间隔或更换User-Agent。
• 方案2的yfinance库会持续更新以适配Yahoo的变化，是更稳定的选择。

内容的提问来源于stack exchange，提问作者Colin Zhong