如何避免Selenium在新内容加载完成前过快点击‘Show More’按钮？

你的核心问题是脚本在新内容未完全渲染时就尝试再次点击，导致无效操作。以下是两种更可靠的优化方案：

方案1：等待Show More按钮重新可点击（最直接） #

点击按钮后，按钮会消失并被加载动画替代，加载完成后按钮会重新出现。我们可以通过等待按钮从不可见到重新可点击，来确保加载完成：

from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import TimeoutException, NoSuchElementException

service = Service("C:\\Users\\sddd\\Downloads\\chromedriver-win64\\chromedriver.exe")
driver = webdriver.Chrome(service=service)
driver.get("https://oddspedia.com/football")

try:
    # 接受Cookie
    cookie_button = WebDriverWait(driver, 5).until(
        EC.element_to_be_clickable((By.XPATH, '//button[@class="cookie-popup__btn"]'))
    )
    cookie_button.click()
    print("Cookies accepted.")

    while True:
        try:
            # 等待Show More按钮可点击
            show_more_button = WebDriverWait(driver, 5).until(
                EC.element_to_be_clickable((By.XPATH, '//div[@class="ml-show-more-btn"]/button'))
            )
            show_more_button.click()
            print("Clicked 'Show more' button.")

            # 等待按钮消失（加载开始）
            WebDriverWait(driver, 10).until(
                EC.invisibility_of_element_located((By.XPATH, '//div[@class="ml-show-more-btn"]/button'))
            )
            print("Loading started, waiting for completion...")

            # 等待按钮重新可点击（加载完成）
            WebDriverWait(driver, 10).until(
                EC.element_to_be_clickable((By.XPATH, '//div[@class="ml-show-more-btn"]/button'))
            )
            print("Loading completed, ready for next click.")

        except TimeoutException:
            print("No more 'Show more' button or loading timed out.")
            break

except NoSuchElementException:
    print("Element not found.")
finally:
    driver.quit()

方案2：等待赛事条目数量增加（最可靠） #

通过对比点击前后的赛事条目数量，确保新数据已渲染到DOM中，这是判断内容加载完成最准确的方式：

from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import TimeoutException, NoSuchElementException

service = Service("C:\\Users\\sddd\\Downloads\\chromedriver-win64\\chromedriver.exe")
driver = webdriver.Chrome(service=service)
driver.get("https://oddspedia.com/football")

# 定义赛事条目定位器（根据网站实际DOM调整，这里假设是ml-event类）
EVENT_ITEM_LOCATOR = (By.CLASS_NAME, 'ml-event')

try:
    # 接受Cookie
    cookie_button = WebDriverWait(driver, 5).until(
        EC.element_to_be_clickable((By.XPATH, '//button[@class="cookie-popup__btn"]'))
    )
    cookie_button.click()
    print("Cookies accepted.")

    while True:
        try:
            # 等待Show More按钮可点击
            show_more_button = WebDriverWait(driver, 5).until(
                EC.element_to_be_clickable((By.XPATH, '//div[@class="ml-show-more-btn"]/button'))
            )

            # 记录当前赛事数量
            before_count = len(driver.find_elements(*EVENT_ITEM_LOCATOR))
            show_more_button.click()
            print(f"Clicked 'Show more' button. Current event count: {before_count}")

            # 等待赛事数量增加，超时10秒
            WebDriverWait(driver, 10).until(
                lambda d: len(d.find_elements(*EVENT_ITEM_LOCATOR)) > before_count
            )
            after_count = len(driver.find_elements(*EVENT_ITEM_LOCATOR))
            print(f"Loading completed. New event count: {after_count}")

        except TimeoutException:
            print("No more 'Show more' button or loading timed out.")
            break

except NoSuchElementException:
    print("Element not found.")
finally:
    driver.quit()

关键优化点说明 #

• 移除了固定sleep(1)，改用动态等待，既避免不必要的等待，也确保操作时机准确。
• 方案1利用按钮的可见性变化判断加载状态，适配网站的交互逻辑；方案2直接验证数据加载结果，可靠性更高。
• 所有等待操作都设置了合理超时时间，避免脚本无限挂起。

内容的提问来源于stack exchange，提问作者Istvan