解决Selenium点击Yahoo Finance运营费用按钮的ElementClickInterceptedException

问题背景 #

爬取Yahoo Finance苹果公司利润表（链接）时，点击Operating Expense按钮展开研发费用数据，遭遇ElementClickInterceptedException，提示点击被全屏遮罩层（lightbox-wrapper类的div）拦截。

当前代码：

def clickOperatingExpense(url):
        options = Options()
        options.add_argument('--headless')
        driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()), options=options)
        driver.get(url)
        # 此处触发ElementClickInterceptedException
        WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, '//*[@id="Col1-1-Financials-Proxy"]/section/div[4]/div[1]/div[1]/div[2]/div[4]/div[1]/div[1]/div[1]/button'))).click()

错误信息：

selenium.common.exceptions.ElementClickInterceptedException: Message: element click intercepted: Element <button aria-label="Operating Expense" class="P(0) M(0) Va(m) Bd(0) Fz(s) Mend(2px) tgglBtn">...</button> is not clickable at point (39, 592). Other element would receive the click: <div tabindex="1" class="lightbox-wrapper Ta(c) Pos(f) T(0) Start(0) H(100%) W(100%) Bgc($modalBackground) Ovy(a) Z(50) Op(1)">...</div>

解决方法 #

1. 处理拦截的遮罩层 #

错误中的lightbox-wrapper是网站弹出的全屏遮罩（通常是Cookie提示或登录弹窗），先通过JavaScript隐藏它：

def clickOperatingExpense(url):
    from selenium import webdriver
    from selenium.webdriver.chrome.options import Options
    from selenium.webdriver.chrome.service import Service
    from webdriver_manager.chrome import ChromeDriverManager
    from selenium.webdriver.common.by import By
    from selenium.webdriver.support.ui import WebDriverWait
    from selenium.webdriver.support import expected_conditions as EC

    options = Options()
    options.add_argument('--headless')
    options.add_argument('--window-size=1920,1080')  # 模拟正常窗口大小，避免元素因视口问题不可见
    driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()), options=options)
    driver.get(url)
    
    # 尝试隐藏遮罩层
    try:
        overlay = WebDriverWait(driver, 5).until(EC.presence_of_element_located((By.CLASS_NAME, 'lightbox-wrapper')))
        driver.execute_script("arguments[0].style.display = 'none';", overlay)
    except:
        # 无遮罩层时跳过
        pass
    
    # 定位并点击目标按钮
    target_btn = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, '//button[@aria-label="Operating Expense"]')))
    target_btn.click()

2. 用JavaScript强制点击按钮 #

如果遮罩层处理后仍有拦截，直接通过JS触发按钮的点击事件，绕过元素可见性检查：

# 替换原来的target_btn.click()为以下代码
driver.execute_script("arguments[0].click();", target_btn)

3. 优化元素定位 #

原XPATH依赖冗长的DOM层级，极易失效，改用基于aria-label的稳定定位：
By.XPATH, '//button[@aria-label="Operating Expense"]'
或结合类名：
By.CSS_SELECTOR, 'button.tgglBtn[aria-label="Operating Expense"]'

内容的提问来源于stack exchange，提问作者AJ Goudel