Selenium爬取Betika网站时NoSuchElementException报错求助

爬取Betika博彩网站（https://www.betika.com/en-ke/）的主胜、平局、客胜赔率时，前10条数据能正常获取，但循环到第11条时抛出NoSuchElementException错误。已在driver.get(url)后添加隐式等待，问题仍未解决。

错误信息：

selenium.common.exceptions.NoSuchElementException:
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html[@class='no-js dark']/body/div[@class='app']/main/div[@class='mobile']/div[@class='mobile-router_view-container']/div[@class='contain-overflow']/div[@class='matches__container']/div[@class='matches']/div[@class='vue-pull-to-wrapper']/div[@class='scroll-container']/div[@class='prebet-match'][11]/div[@class='prebet-match__odd-market__container']/div[@class='prebet-match__teams']/span[@class='prebet-match__teams__home']"}

原代码：

from selenium import webdriver
from selenium.webdriver.common.by import By

option = webdriver.ChromeOptions()
option.add_argument('headless')

PATH = "./chromedriver.exe"
url = 'https://www.betika.com/en-ke/'

driver = webdriver.Chrome(PATH, options=option)

driver.get(url)


home_teams = []
away_teams = []
home_odds = []
draw = []
away_odds = []


for n in range(1,20):
    home_teams.append(driver.find_element(By.XPATH, f"/html[@class='no-js dark']/body/div[@class='app']/main/div[@class='mobile']/div[@class='mobile-router_view-container']/div[@class='contain-overflow']/div[@class='matches__container']/div[@class='matches']/div[@class='vue-pull-to-wrapper']/div[@class='scroll-container']/div[@class='prebet-match'][{n}]/div[@class='prebet-match__odd-market__container']/div[@class='prebet-match__teams']/span[@class='prebet-match__teams__home']").get_attribute('innerHTML'))
    away_teams.append(driver.find_element(By.XPATH, f"/html[@class='no-js dark']/body/div[@class='app']/main/div[@class='mobile']/div[@class='mobile-router_view-container']/div[@class='contain-overflow']/div[@class='matches__container']/div[@class='matches']/div[@class='vue-pull-to-wrapper']/div[@class='scroll-container']/div[@class='prebet-match'][{n}]/div[@class='prebet-match__odd-market__container']/div[@class='prebet-match__teams']/span[2]").get_attribute('innerHTML'))
    home_odds.append(float(driver.find_element(By.XPATH, f"/html[@class='no-js dark']/body/div[@class='app']/main/div[@class='mobile']/div[@class='mobile-router_view-container']/div[@class='contain-overflow']/div[@class='matches__container']/div[@class='matches']/div[@class='vue-pull-to-wrapper']/div[@class='scroll-container']/div[@class='prebet-match'][{n}]/div[@class='prebet-match__odd-market__container']/div[@class='prebet-match__odds__container']/div[@class='prebet-match__odds']/button[@class='prebet-match__odd'][1]/span[@class='prebet-match__odd__odd-value bold']").get_attribute('innerHTML')))
    draw.append(float(driver.find_element(By.XPATH, f"/html[@class='no-js dark']/body/div[@class='app']/main/div[@class='mobile']/div[@class='mobile-router_view-container']/div[@class='contain-overflow']/div[@class='matches__container']/div[@class='matches']/div[@class='vue-pull-to-wrapper']/div[@class='scroll-container']/div[@class='prebet-match'][{n}]/div[@class='prebet-match__odd-market__container']/div[@class='prebet-match__odds__container']/div[@class='prebet-match__odds']/button[@class='prebet-match__odd'][2]/span[@class='prebet-match__odd__odd-value bold']").get_attribute('innerHTML')))
    away_odds.append(float(driver.find_element(By.XPATH, f"/html[@class='no-js dark']/body/div[@class='app']/main/div[@class='mobile']/div[@class='mobile-router_view-container']/div[@class='contain-overflow']/div[@class='matches__container']/div[@class='matches']/div[@class='vue-pull-to-wrapper']/div[@class='scroll-container']/div[@class='prebet-match'][{n}]/div[@class='prebet-match__odd-market__container']/div[@class='prebet-match__odds__container']/div[@class='prebet-match__odds']/button[@class='prebet-match__odd'][3]/span[@class='prebet-match__odd__odd-value bold']").get_attribute('innerHTML')))

核心问题分析 #

1. 绝对XPath不可靠：使用从根节点开始的绝对路径，页面结构只要有微小变化（比如新增/移除节点）就会导致定位失败；且第11条之后的内容可能是滚动加载的，初始状态下并未渲染到DOM中。
2. 隐式等待局限性：隐式等待只针对全局元素查找，无法处理滚动加载的动态内容，也不能确保特定元素完全可交互。

具体修复步骤 #

1. 使用相对XPath定位所有赛事节点 #

先获取页面中所有的prebet-match元素，再遍历每个元素获取子节点信息，避免依赖动态索引的绝对路径。

2. 处理滚动加载 #

通过JavaScript滚动到页面底部，触发更多赛事内容的加载；或者遍历到某个节点时，滚动到该节点位置确保其渲染。

3. 替换为显式等待 #

针对每个需要获取的元素，使用显式等待确保元素存在后再操作，提升稳定性。

4. 添加异常处理 #

遍历过程中捕获NoSuchElementException，避免程序因单个元素定位失败而崩溃。

修改后的代码示例 #

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import NoSuchElementException, TimeoutException

option = webdriver.ChromeOptions()
option.add_argument('headless')
# 无头模式下需设置窗口大小，避免元素因视口问题无法定位
option.add_argument('window-size=1920x1080')

PATH = "./chromedriver.exe"
url = 'https://www.betika.com/en-ke/'

driver = webdriver.Chrome(PATH, options=option)
wait = WebDriverWait(driver, 10)  # 显式等待超时时间10秒

driver.get(url)

home_teams = []
away_teams = []
home_odds = []
draw = []
away_odds = []

try:
    # 先滚动到页面底部，加载更多内容
    driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
    # 等待赛事列表加载完成
    wait.until(EC.presence_of_all_elements_located((By.CLASS_NAME, 'prebet-match')))
    
    # 获取所有赛事节点
    match_elements = driver.find_elements(By.CLASS_NAME, 'prebet-match')
    
    # 遍历前20个（如果存在的话）
    for match in match_elements[:20]:
        try:
            # 获取主队名称
            home_team = match.find_element(By.CLASS_NAME, 'prebet-match__teams__home').text
            home_teams.append(home_team)
            
            # 获取客队名称
            away_team = match.find_element(By.XPATH, './/div[@class="prebet-match__teams"]/span[2]').text
            away_teams.append(away_team)
            
            # 获取三个赔率
            odds_buttons = match.find_elements(By.CLASS_NAME, 'prebet-match__odd')
            if len(odds_buttons) >=3:
                home_odd = float(odds_buttons[0].find_element(By.CLASS_NAME, 'prebet-match__odd__odd-value').text)
                draw_odd = float(odds_buttons[1].find_element(By.CLASS_NAME, 'prebet-match__odd__odd-value').text)
                away_odd = float(odds_buttons[2].find_element(By.CLASS_NAME, 'prebet-match__odd__odd-value').text)
                
                home_odds.append(home_odd)
                draw.append(draw_odd)
                away_odds.append(away_odd)
        except NoSuchElementException:
            # 跳过当前赛事，继续下一个
            continue
finally:
    driver.quit()

# 打印结果
for i in range(len(home_teams)):
    print(f"赛事{i+1}: {home_teams[i]} vs {away_teams[i]} | 主胜:{home_odds[i]} 平局:{draw[i]} 客胜:{away_odds[i]}")

代码说明 #

• 相对定位：通过match.find_element从当前赛事节点出发查找子元素，避免绝对路径的脆弱性。
• 滚动加载：用window.scrollTo触发页面底部加载，确保更多赛事内容渲染。
• 显式等待：WebDriverWait等待赛事列表加载完成，确保元素存在后再操作。
• 异常捕获：遍历单个赛事时捕获异常，跳过有问题的赛事，保证程序运行。

内容的提问来源于stack exchange，提问作者Frosting