Python Selenium webdriver关闭前如何循环采集多轮次赛事数据

你的代码和需求不匹配的核心问题如下：

• URL拼接错误：遍历赛事列表时，你错误传入了整个赛事列表ytournaments做域名拼接，实际应该传入当前循环对应的单条赛事路径变量tournamentid
• Webdriver生命周期不符合要求：当前driver在所有循环外全局初始化，只启动1次。需要把driver启动、窗口最大化的逻辑移到单个赛事的循环内部，等该赛事全部轮次抓取完成后再关闭实例，即可实现「单赛事采集完释放实例、新赛事启动新实例」的逻辑
• 路径转义隐患：ChromeDriver的Windows路径前加原生字符串标记r，避免反斜杠触发转义报错
• 冗余代码清理：移除全局的driver初始化逻辑、循环外多余的driver.quit()调用，同时删除代码中定义后未使用的无效变量

from selenium import webdriver
import time
import pandas as pd
import numpy as np
from bs4 import BeautifulSoup


# 加r标记避免Windows路径反斜杠转义问题
PATH = r"C:\Program Files (x86)\Chrome\chromedriver_win32\chromedriver.exe"
# 赛事列表存储所有拿到的赛事详情页href路径
ytournaments = ['/dpworld-tour/abu-dhabi-hsbc-championship-2021/']
roundids = [1, 2, 3, 4]
base_domain = "https://www.europeantour.com"

for tournamentid in ytournaments:
    # 每个赛事启动全新webdriver实例
    driver = webdriver.Chrome(PATH)
    driver.maximize_window()
    
    for roundid in roundids:
        # 用循环变量tournamentid拼接单轮次页面URL
        target_url = f"{base_domain}{tournamentid}leaderboard?holebyhole=true&round={roundid}"
        driver.get(target_url)
        time.sleep(5)
        html = driver.page_source
        soup = BeautifulSoup(html, 'lxml')
        
        Tour = 'European Tour'
        Year = '2021'
        
        tournamentm = soup.find('h1', class_='event-hero__title').text
        tournament = tournamentm.strip()
        
        coursem = soup.find('p', class_='event-hero__location').text
        course = coursem.strip()
        
        datem = soup.find('p', class_='event-hero__date').text
        date = datem.strip()
        
        dfs = pd.read_html(driver.page_source)
        ndf = np.squeeze(dfs)
        data = pd.DataFrame(ndf)
        
        data["tournament"] = tournament
        data["course"] = course
        data["date"] = date
        data["roundid"] = roundid
        data["Tour"] = Tour
        data["Year"] = Year
        
        filename = f'{tournament}_{roundid}_{Year}.csv'
        data.to_csv(filename)
    
    # 当前赛事所有轮次采集完成，关闭当前webdriver实例
    driver.quit()

注：如果后续需要扩展采集更多赛事，只需要把新拿到的赛事详情页href追加到ytournaments列表即可，代码会自动按逻辑启动新实例完成采集。

内容的提问来源于stack exchange，提问作者TSned3