如何用Python从Rotowire抓取MLB每日首发投手与打者阵容？

以下是实现你需求的完整代码，通过requests获取页面内容，BeautifulSoup解析HTML，最终生成首发投手和首发打者两个DataFrame：

import pandas as pd
import requests
from bs4 import BeautifulSoup

# 设置请求头，模拟浏览器访问以避免被拦截
headers = {
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36"
}
url = "https://www.rotowire.com/baseball/daily-lineups.php"
response = requests.get(url, headers=headers)
soup = BeautifulSoup(response.content, "html.parser")

# 初始化存储投手和打者数据的列表
pitcher_data = []
hitter_data = []

# 遍历每个赛事区块
for game in soup.select(".lineup"):
    # 提取赛事日期和时间
    game_top = game.select_one(".lineup__top").get_text(strip=True)
    # 拆分日期和时间（格式如"Today · 1:05 PM ET"）
    date_part, time_part = game_top.split(" · ")
    game_date = date_part
    game_time = time_part
    
    # 获取对阵双方球队
    teams = game.select(".lineup__team")
    away_team = teams[0].get_text(strip=True)
    home_team = teams[1].get_text(strip=True)
    
    # 遍历两队的阵容区块
    team_wrappers = game.select(".lineup__team-wrapper")
    for idx, wrapper in enumerate(team_wrappers):
        current_team = away_team if idx == 0 else home_team
        
        # 提取首发投手信息
        pitcher = wrapper.select_one(".lineup__player.is-pitcher")
        if pitcher:
            pitcher_name = pitcher.select_one(".lineup__player-name").get_text(strip=True)
            pitcher_hand = pitcher.select_one(".lineup__player-hand").get_text(strip=True)
            pitcher_data.append({
                "日期": game_date,
                "比赛时间": game_time,
                "投手姓名": pitcher_name,
                "球队": current_team,
                "投球惯用手": pitcher_hand
            })
        
        # 提取首发打者信息
        hitters = wrapper.select(".lineup__player.is-batter")
        for hitter in hitters:
            hitter_order = hitter.select_one(".lineup__player-order").get_text(strip=True)
            hitter_name = hitter.select_one(".lineup__player-name").get_text(strip=True)
            hitter_pos = hitter.select_one(".lineup__player-pos").get_text(strip=True)
            hitter_hand = hitter.select_one(".lineup__player-hand").get_text(strip=True)
            hitter_data.append({
                "日期": game_date,
                "比赛时间": game_time,
                "打者姓名": hitter_name,
                "球队": current_team,
                "位置": hitter_pos,
                "打击顺序": hitter_order,
                "打击惯用手": hitter_hand
            })

# 转换为DataFrame
pitcher_df = pd.DataFrame(pitcher_data)
hitter_df = pd.DataFrame(hitter_data)

# 打印结果示例
print("首发投手DataFrame：")
print(pitcher_df.head())
print("\n首发打者DataFrame：")
print(hitter_df.head())

代码说明 #

• 请求头设置：添加User-Agent模拟浏览器访问，避免被网站反爬机制拦截
• 赛事区块遍历：通过.lineup选择器定位每个独立赛事，确保覆盖当日所有比赛
• 日期时间提取：从.lineup__top区块拆分日期和时间，若需要标准化日期格式，可结合datetime模块替换Today为当前日期
• 投手信息解析：定位.is-pitcher类的元素，提取姓名和投球惯用手
• 打者信息解析：遍历.is-batter类的元素，提取打击顺序、姓名、位置和打击惯用手
• 数据存储：将提取的信息存入列表，最终转换为Pandas DataFrame，方便后续分析或导出

内容的提问来源于stack exchange，提问作者StLouisO