Python批量获取股票与股息数据失败，求解决方案（附代码）

方法一：修复requests+BeautifulSoup爬取Yahoo Finance的问题 #

原代码失败核心原因：

• Yahoo Finance存在基础反爬机制，直接请求会被拦截并返回验证页面
• 页面元素的class属性已更新，原选择器无法定位目标内容

修复后的代码：

import requests
from bs4 import BeautifulSoup

# 模拟浏览器请求头，绕过基础反爬拦截
HEADERS = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36'
}

def get_stock_price(stock):
    url = f'https://finance.yahoo.com/quote/{stock}/'
    response = requests.get(url, headers=HEADERS)
    soup = BeautifulSoup(response.text, 'html.parser')
    # 用语义化属性定位股价元素
    price_element = soup.find('fin-streamer', {'data-field': 'regularMarketPrice'})
    if price_element:
        price = price_element.text
        return float(price.replace(',', ''))
    return None

def get_dividend_data(stock):
    url = f'https://finance.yahoo.com/quote/{stock}/dividends?p={stock}'
    response = requests.get(url, headers=HEADERS)
    soup = BeautifulSoup(response.text, 'html.parser')
    # 用CSS选择器定位股息表格中最新的股息金额
    dividend_element = soup.select_one('table tbody tr:first-child td:nth-child(2)')
    if dividend_element:
        dividend = dividend_element.text
        return float(dividend.replace(',', ''))
    return None

stocks = ['MSFT', 'LTC', 'AGNC', 'GLAD', 'HRZN', 'ARCC', 'HASI', 'SBR']
for stock in stocks:
    price = get_stock_price(stock)
    dividend = get_dividend_data(stock)
    print(f"Stock: {stock}, Price: {price}, Dividend: {dividend}")

修复说明：

• 添加User-Agent请求头，模拟真实浏览器访问
• 更新元素定位逻辑：使用fin-streamer标签的data-field属性定位股价，用CSS选择器定位股息表格数据

方法二：修复yfinance库的使用问题 #

原代码失败核心原因：

• regularMarketPrice字段稳定性差，部分场景下未返回数据
• 直接通过字典键取值触发KeyError，未做容错处理
• 存在冗余循环，重复请求数据导致效率低下

修复后的代码：

import yfinance as yf

stock_data = {}
ticker_symbols_monthly = ['MSFT', 'LTC','AGNC', 'GLAD', 'HRZN', 'ARCC', 'HASI', 'SBR']

def get_stock_data():
    for ticker_symbol in ticker_symbols_monthly:
        stock = yf.Ticker(ticker_symbol)
        info = stock.info
        # 用更稳定的currentPrice字段替代regularMarketPrice
        price = info.get('currentPrice', 0)
        dividend_yield = info.get('dividendYield', 0)
        market_cap = info.get('marketCap', 0)

        print(f'Stock for {ticker_symbol}')
        print(f'Stock price: {price}')
        print(f'Dividend yield: {dividend_yield}')
        print(f'Market capitalization: {market_cap}')
        print("\n")

        # 容错处理股息收益率为None的情况
        annual_dividend = dividend_yield * price if dividend_yield else 0
        stock_data[ticker_symbol] = (price, annual_dividend)

get_stock_data()
# 输出整理后的结构化数据
print("整理后的股票数据：")
for ticker, (price, annual_div) in stock_data.items():
    print(f"{ticker}: 股价={price}, 年度股息={annual_div}")

修复说明：

• 移除冗余循环，单次请求即可获取所有所需数据
• 使用currentPrice字段获取股价，稳定性优于原字段
• 优化容错逻辑，避免KeyError和计算错误
• 简化年度股息计算逻辑，提升可读性

内容的提问来源于stack exchange，提问作者David Flenaugh