Selenium爬取时Chrome瞬开瞬关且matches变量为空的问题求助

一、解决Chrome浏览器立即关闭的问题 #

核心原因 #

• 脚本执行完毕后driver对象被销毁，浏览器默认自动关闭
• 手动安装ChromeDriver易出现版本与Chrome浏览器不匹配的问题，导致启动异常

修复方案 #

1. 添加detach实验选项，强制浏览器在脚本结束后保持打开状态
2. 用webdriver-manager自动管理ChromeDriver，彻底规避版本不匹配问题

代码修改片段 #

options = webdriver.ChromeOptions()
options.add_experimental_option("excludeSwitches", ["enable-logging"])
# 新增：防止浏览器自动关闭
options.add_experimental_option("detach", True)
# 新增：自动适配ChromeDriver版本，无需手动安装
driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()), options=options)

二、解决matches变量为空的问题 #

核心原因 #

• 点击按钮后页面未完成渲染就直接查找元素，此时比赛表格还未加载
• 直接用By.TAG_NAME, "tr"会匹配页面所有<tr>元素，包含非比赛表格的无效元素，且未等待目标元素加载完成

修复方案 #

1. 使用显式等待WebDriverWait，等待比赛表格的<tr>元素加载完毕后再获取
2. 精准定位比赛表格内的<tr>元素（通过表格class缩小范围）

代码修改步骤 #

首先导入显式等待依赖：

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

然后替换原有的matches获取代码：

# 等待比赛表格加载，最多等待10秒，精准定位目标表格内的行
matches = WebDriverWait(driver, 10).until(
    EC.presence_of_all_elements_located((By.XPATH, "//table[@class='table']/tbody/tr"))
)

同时，所有交互操作也建议添加显式等待，避免页面加载慢导致的元素定位失败：

# 点击All matches按钮
all_matches_button = WebDriverWait(driver, 10).until(
    EC.element_to_be_clickable((By.XPATH, '//label[@analytics-event="All matches"]'))
)
all_matches_button.click()

# 选择赛季下拉框
season = WebDriverWait(driver, 10).until(
    EC.element_to_be_clickable((By.XPATH, '//*[@id="season"]'))
)
season.click()

# 选择2022赛季
s_2022 = WebDriverWait(driver, 10).until(
    EC.element_to_be_clickable((By.XPATH, '//*[@id="season"]/option[2]'))
)
s_2022.click()

完整修正代码 #

from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

options = webdriver.ChromeOptions()
options.add_experimental_option("excludeSwitches", ["enable-logging"])
options.add_experimental_option("detach", True)
driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()), options=options)
driver.get("https://www.adamchoi.co.uk/overs/detailed")

# 点击All matches按钮
all_matches_button = WebDriverWait(driver, 10).until(
    EC.element_to_be_clickable((By.XPATH, '//label[@analytics-event="All matches"]'))
)
all_matches_button.click()

# 选择赛季下拉框并点击2022选项
season = WebDriverWait(driver, 10).until(
    EC.element_to_be_clickable((By.XPATH, '//*[@id="season"]'))
)
season.click()

s_2022 = WebDriverWait(driver, 10).until(
    EC.element_to_be_clickable((By.XPATH, '//*[@id="season"]/option[2]'))
)
s_2022.click()

# 等待比赛表格加载并获取所有行
matches = WebDriverWait(driver, 10).until(
    EC.presence_of_all_elements_located((By.XPATH, "//table[@class='table']/tbody/tr"))
)

date = []
home_team = []
score = []
away_team = []

print(f"找到{len(matches)}场比赛")

for match in matches:
    date.append(match.find_element(By.XPATH, "./td[1]").text)
    home = match.find_element(By.XPATH, "./td[2]").text
    print(home)
    score.append(match.find_element(By.XPATH, "./td[3]").text)
    away_team.append(match.find_element(By.XPATH, "./td[4]").text)

内容的提问来源于stack exchange，提问作者rosa