使用Selenium Python爬取动态网页时如何规避ElementClickInterceptedException错误

问题场景 #

爬取网页表格时，通过Selenium循环点击下拉菜单中的赛季选项切换页面，不定期触发ElementClickInterceptedException，错误均来自season_tags[i].click()代码行，循环执行次数不稳定（有时12次，有时仅6次）。

原代码片段 #

for i in range(0,len(season_tags)):

    """ 
    Step 1: Get to the correct Page

    """
    #Click to open the season dropdown
    driver.find_elements(By.CLASS_NAME,'current')[1].click()
    wait = WebDriverWait(driver,10)# wait for dropdown to apear
    wait.until(EC.presence_of_element_located((By.CLASS_NAME,'current')))
    
    
    #Click to chose a season
    season_tags[i].click()

    """ 
    Step 2: Scrape the data from the page
            - Format the data into a table
    """
    #Wait for table to appear
    driver.implicitly_wait(5)

    # Find table data and split it up
    table_data = driver.find_element(By.CLASS_NAME,'tableBodyContainer').text.split('\n')

    #Extract the data
    team_names = table_data[1::3]
    num_data = table_data[2::3]

    # Put the data into a dataframe
    data = pd.DataFrame([i.split(' ')for i in num_data])
    data['Teams'] = team_names
    print(season_tags[i].text)

    #Concatenate the table data
    frames.append(data)

报错信息 #

ElementClickInterceptedException          Traceback (most recent call last)
Input In [203], in <cell line: 4>()
     14 driver.implicitly_wait(5)
     16 #Click to chose a season
---> 17 season_tags[i].click()
     19 """ 
     20 Step 2: Scrape the data from the page
     21         - Format the data into a table
     22 """
     23 #Wait for table to appear

File ~\Anaconda\lib\site-packages\selenium\webdriver\remote\webelement.py:88, in WebElement.click(self)
     86 def click(self) -> None:
     87     """Clicks the element."""
---> 88     self._execute(Command.CLICK_ELEMENT)

File ~\Anaconda\lib\site-packages\selenium\webdriver\remote\webelement.py:396, in WebElement._execute(self, command, params)
    394     params = {}
    395 params['id'] = self._id
---> 396 return self._parent.execute(command, params)

File ~\Anaconda\lib\site-packages\selenium\webdriver\remote\webdriver.py:429, in WebDriver.execute(self, driver_command, params)
    427 response = self.command_executor.execute(driver_command, params)
    428 if response:
---> 429     self.error_handler.check_response(response)
    430     response['value'] = self._unwrap_value(
    431         response.get('value', None))
    432     return response

File ~\Anaconda\lib\site-packages\selenium\webdriver\remote\errorhandler.py:243, in ErrorHandler.check_response(self, response)
    241         alert_text = value['alert'].get('text')
    242     raise exception_class(message, screen, stacktrace, alert_text)  # type: ignore[call-arg]  # mypy is not smart enough here
---> 243 raise exception_class(message, screen, stacktrace)

ElementClickInterceptedException: Message: element click intercepted: Element <li role="option" tabindex="0" data-option-name="..." data-option-id="210" data-option-index="5" class="">2018/19</li> is not clickable at point (245, 437). Other element would receive the click: <li role="option" tabindex="0" data-option-name="..." data-option-id="274" data-option-index="4" class="">2019/20</li>
  (Session info: chrome=106.0.5249.119)
Stacktrace:
Backtrace:

解决方案 #

1. 优化等待逻辑+重新定位元素 #

原代码复用提前获取的season_tags元素列表，页面切换后元素可能失效（StaleElementReferenceException），且等待仅判断元素存在而非可点击。修改思路：

• 提前保存所有赛季的文本内容
• 每次循环打开下拉菜单后，重新定位当前赛季元素
• 等待元素可点击而非仅存在

from selenium.webdriver.support import expected_conditions as EC

# 提前获取所有赛季的文本，避免元素失效
season_tags_text = [tag.text for tag in season_tags]

for i in range(len(season_tags_text)):
    # 打开赛季下拉菜单
    dropdown = driver.find_elements(By.CLASS_NAME, 'current')[1]
    dropdown.click()
    
    wait = WebDriverWait(driver, 10)
    # 等待下拉菜单展开，且目标元素可点击
    target_season = wait.until(
        EC.element_to_be_clickable(
            (By.XPATH, f"//li[@role='option' and text()='{season_tags_text[i]}']")
        )
    )
    target_season.click()
    
    # 等待表格加载完成（替换原隐式等待）
    wait.until(EC.presence_of_element_located((By.CLASS_NAME, 'tableBodyContainer')))
    
    # 后续爬取逻辑不变
    table_data = driver.find_element(By.CLASS_NAME,'tableBodyContainer').text.split('\n')
    team_names = table_data[1::3]
    num_data = table_data[2::3]
    data = pd.DataFrame([i.split(' ')for i in num_data])
    data['Teams'] = team_names
    print(season_tags_text[i])
    frames.append(data)

2. 使用JavaScript强制点击（备选方案） #

若元素因页面布局遮挡无法正常点击，可通过JS直接触发元素的点击事件，绕过页面交互拦截：

# 替换原有的 season_tags[i].click()
driver.execute_script("arguments[0].click();", season_tags[i])

注意：此方法会跳过页面的交互校验，可能导致页面状态异常，仅在方法1无效时使用。

3. 滚动元素至视口后点击 #

若元素不在当前视口范围内，点击时可能被其他元素遮挡，先滚动到元素位置再点击：

# 滚动到目标元素位置
driver.execute_script("arguments[0].scrollIntoView(true);", season_tags[i])
# 等待元素可点击后再点击
wait = WebDriverWait(driver, 10)
wait.until(EC.element_to_be_clickable(season_tags[i])).click()

4. 避免混用隐式等待与显式等待 #

原代码同时使用了WebDriverWait（显式等待）和driver.implicitly_wait(5)（隐式等待），两者混用会导致等待时间不可控，建议统一使用显式等待。

内容的提问来源于stack exchange，提问作者Efan Mutembo