Yahoo网页爬取仅返回100条数据问题求助（API替代方案）

Fixing the 100-Record Limit in Yahoo Finance Historical Data Scraping #

Hey there! I’ve run into this exact issue before when pulling historical stock data from Yahoo Finance—let me walk you through how to get the full year’s data instead of just the default 100 records.

1. Explicitly Define Your Date Range with Timestamps #

Yahoo Finance’s underlying data endpoint defaults to returning 100 records unless you specify a full date range. The key here is to use the period1 and period2 parameters, which take Unix timestamps (in seconds) for your desired start and end dates.

For example, if you want 2021’s full data, convert the start (Jan 1, 2021) and end (Dec 31, 2021) dates to timestamps, then include them in your request. Here’s how you can calculate those timestamps in Python:

import datetime

# Set your target date range
start_date = datetime.datetime(2021, 1, 1)
end_date = datetime.datetime(2021, 12, 31)

# Convert to Unix timestamps (in seconds)
period1 = int(start_date.timestamp())
period2 = int(end_date.timestamp())

When building your request, add these parameters along with interval=1d (for daily data) and events=history (to include dividends/splits, which you mentioned seeing). This will pull every daily record in your date range, not just the first 100.

2. Adjust the count Parameter (If Needed) #

A full year of trading data includes roughly 252 days, plus any dividend or split events. To make sure you capture all of them, explicitly set the count parameter to a value larger than this total—count=300 works perfectly for most annual datasets.

3. Pagination for Longer Timeframes #

If you’re scraping multiple years of data and need to break it into chunks, use the offset parameter to paginate:

• First pull: offset=0&count=100
• Next pull: offset=100&count=100
• Repeat until the returned dataframe has fewer than 100 records, then concatenate all the dataframes together.

Quick Example Request URL #

Here’s what a full request URL might look like for Apple’s 2021 data:
https://query1.finance.yahoo.com/v7/finance/download/AAPL?period1=1609459200&period2=1640995200&interval=1d&events=history&includeAdjustedClose=true

This will return every daily entry (including dividends) for the entire year, not just 100 records.

内容的提问来源于stack exchange，提问作者dborger