Selenium无法定位Cookie弹窗按钮问题排查

问题背景 #

我尝试抓取奔驰官网（https://shop.mercedes-benz.com/tr-tr/shop/vehicle/srp/used?sort=relevance&assortment=vehicle）的车辆名称、价格等信息。网站初始仅加载12辆车，需要点击「Daha fazla otomobil göster（显示更多车辆）」按钮加载更多内容，但此时会弹出全屏Cookie弹窗，需点击「Yalnızca zorunlu çerezleri kabul et（仅接受必要Cookie）」按钮关闭它。运行以下代码时出现错误：

from selenium import webdriver
import selenium
import time

def close_banner(browser):
    browser.find_element_by_class_name('button--only-technically-necessary').click()

url = "https://shop.mercedes-benz.com/tr-tr/shop/vehicle/srp/used?sort=relevance&assortment=vehicle"

opt = webdriver.ChromeOptions()

opt.add_argument('--start-maximized')
driver = webdriver.Chrome(options=opt)

driver.get(url)
time.sleep(10)
btn = driver.find_element_by_xpath('//*[@id="app"]/div[1]/main/div[2]/div[1]/div[2]/div[2]/div[4]/button')
while True:
    try:
        btn.click()
        break
    except selenium.common.exceptions.WebDriverException:
        close_banner(driver)

Cookie弹窗按钮HTML结构 #

<wb7-button class="button button--only-technically-necessary wb-button hydrated" data-test="handle-deny-button" variant="primary" size="s" theme="light">Yalnızca zorunlu çerezleri kabul et</wb7-button>

显示更多按钮HTML结构 #

<button data-test-id="dcp-result-pagination__load-more" class="dcp-result-pagination__load-more wb-button wb-button--secondary wb-button--large"> Daha fazla otomobil göster </button>

报错信息 #

selenium.common.exceptions.ElementClickInterceptedException: Message: element click intercepted: Element <button data-test-id="dcp-result-pagination__load-more" class="dcp-result-pagination__load-more wb-button wb-button--secondary wb-button--large">...</button> is not clickable at point (1176, 879). Other element would receive the click: <cmm-cookie-banner settings-id="lNUr7ZitD" variant="A" labels="{"showDetails": "Show details", "hideDetails": "Hide details"}" class="hydrated"></cmm-cookie-banner>

...

During handling of the above exception, another exception occurred:

selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":".button--only-technically-necessary"}

我试过类名、链接文本、CSS选择器等多种定位方式，均未解决问题。请问代码存在什么问题？

问题分析与修复方案 #

1. Cookie按钮定位失败的原因 #

• 你使用的find_element_by_class_name只能匹配单个类名，且自定义标签<wb7-button>可能需要时间完成渲染，直接查找会出现元素未加载的情况。
• 更可靠的定位方式是使用元素的data-test属性（专为测试场景设计，不会轻易变动），而非易变的类名。

2. 逻辑顺序错误 #

代码先尝试点击加载更多按钮，失败后才处理弹窗，但实际上弹窗会提前拦截点击操作。正确逻辑应该是先处理Cookie弹窗，再操作加载更多按钮。

3. 硬等待的不稳定性 #

time.sleep(10)属于硬等待，效率低且无法保证元素完全加载，应替换为Selenium的显式等待，等待元素可点击后再执行操作。

修复后的代码 #

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import selenium

def close_banner(browser):
    try:
        # 用data-test属性定位Cookie按钮，显式等待元素可点击
        cookie_btn = WebDriverWait(browser, 10).until(
            EC.element_to_be_clickable((By.CSS_SELECTOR, 'wb7-button[data-test="handle-deny-button"]'))
        )
        cookie_btn.click()
    except selenium.common.exceptions.TimeoutException:
        # 未检测到弹窗则跳过
        pass

url = "https://shop.mercedes-benz.com/tr-tr/shop/vehicle/srp/used?sort=relevance&assortment=vehicle"

opt = webdriver.ChromeOptions()
opt.add_argument('--start-maximized')
driver = webdriver.Chrome(options=opt)

driver.get(url)

# 优先关闭Cookie弹窗
close_banner(driver)

# 显式等待加载更多按钮可点击
try:
    load_more_btn = WebDriverWait(driver, 10).until(
        EC.element_to_be_clickable((By.CSS_SELECTOR, 'button[data-test-id="dcp-result-pagination__load-more"]'))
    )
    load_more_btn.click()
except selenium.common.exceptions.ElementClickInterceptedException:
    # 弹窗再次出现时重新关闭
    close_banner(driver)
    load_more_btn.click()
except selenium.common.exceptions.TimeoutException:
    print("未找到加载更多按钮或页面加载超时")

关键改进点 #

• 用显式等待替代硬等待，确保元素完全渲染后再操作
• 使用data-test/data-test-id属性定位元素，比类名、XPath更稳定
• 调整执行顺序：先处理弹窗，再操作加载更多按钮
• 增加异常处理，避免单次失败导致程序崩溃

内容的提问来源于stack exchange，提问作者Gokalp