如何解决Python Selenium中的ElementClickInterceptedException异常

我用Python的Selenium框架写了网页爬虫代码：

def click_and_wait_for_additional_info(elem):
    # Click on the store element to navigate to its individual page
    elem.click()
    time.sleep(2)
    
    try:
        features = driver.find_element(By.XPATH,  '//*[@id="QA0Szd"]/div/div/div[1]/div[3]/div/div[1]/div/div/div[2]/div[6]/button/div[2]/div/div').text
    except:
        features = "None"
        
   
    return features

批量处理大量数据时，总会抛出ElementClickInterceptedException异常。

尝试了两种方法：

• a) driver.implicit wait
• b) 改写后的代码：

def click_and_wait_for_additional_info(elem):
    try:
        # Wait for the element to be clickable
        clickable_elem = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, '//div[@class="Nv2PK tH5CWc THOPZb "]')))
        
        # Click on the store element to navigate to its individual page
        clickable_elem.click()
        time.sleep(2)
        
        try:
            features = driver.find_element(By.XPATH,  '//*[@id="QA0Szd"]/div/div/div[1]/div[3]/div/div[1]/div/div/div[2]/div[6]/button/div[2]/div/div').text
        except:
            features = "None"
            
       
        return category

    except ElementClickInterceptedException:
        print("Element click intercepted. Retrying...")
        # Retry clicking on the element
        try:
            clickable_elem = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, '//div[@class="Nv2PK tH5CWc THOPZb "]')))
            clickable_elem.click()
            time.sleep(2)
            # Your code to extract additional info goes here...
        except Exception as e:
            print("Exception occurred:", e)
            return None

    except Exception as e:
        print("Exception occurred:", e)
        return None

现在异常解决了，但出现两个新问题：

• a) 数据偏移
• b) 数据重复

1. 修复核心问题：元素定位不匹配+重试逻辑混乱 #

你改写后的代码完全忽略了传入的elem参数，反而重新用XPATH定位全局元素——这会导致批量处理时，每次都点击页面上第一个符合条件的元素，而非原本要处理的目标元素，直接造成数据重复和偏移。

修正后的核心代码：

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import ElementClickInterceptedException, StaleElementReferenceException

def click_and_wait_for_additional_info(elem, max_retries=3):
    retry_count = 0
    while retry_count < max_retries:
        try:
            # 基于传入的目标elem做等待，而非全局定位
            WebDriverWait(driver, 10).until(EC.element_to_be_clickable(elem))
            # 用JS点击替代普通click，规避多数拦截场景
            driver.execute_script("arguments[0].click();", elem)
            
            # 用显式等待替代固定sleep，确保详情页加载完成
            WebDriverWait(driver, 10).until(
                EC.presence_of_element_located((By.XPATH, '//*[@id="QA0Szd"]/div/div/div[1]/div[3]/div/div[1]/div/div/div[2]/div[6]/button/div[2]/div/div'))
            )
            
            features = driver.find_element(By.XPATH,  '//*[@id="QA0Szd"]/div/div/div[1]/div[3]/div/div[1]/div/div/div[2]/div[6]/button/div[2]/div/div').text
            
            # 爬取完成后必须返回列表页，否则下一次定位会在详情页出错
            driver.back()
            # 等待列表页加载完成，避免元素过期异常
            WebDriverWait(driver, 10).until(
                EC.presence_of_element_located((By.XPATH, '//div[@class="Nv2PK tH5CWc THOPZb "]'))
            )
            return features
        except ElementClickInterceptedException:
            retry_count += 1
            print(f"拦截重试，次数：{retry_count}")
            # 滚动到目标元素位置，避免被弹窗/导航栏遮挡
            driver.execute_script("arguments[0].scrollIntoView(true);", elem)
        except StaleElementReferenceException:
            # 元素引用过期，直接返回None（批量处理时需重新获取元素列表）
            return None
        except Exception as e:
            print(f"异常：{str(e)}")
            return None
    print(f"重试{max_retries}次失败")
    return None

2. 批量处理的关键优化 #

• 每次循环重新获取元素列表：返回列表页后，之前的元素引用会失效（StaleElementReferenceException），复用旧列表会导致定位错误，进而引发数据偏移。示例：

  # 批量处理逻辑
  while True:
      # 每次循环重新拉取当前页的店铺元素列表
      store_elements = driver.find_elements(By.XPATH, '//div[@class="Nv2PK tH5CWc THOPZb "]')
      if not store_elements:
          break
      for elem in store_elements:
          feature = click_and_wait_for_additional_info(elem)
          # 将feature存入数据结构（如列表、字典）
      # 翻页逻辑...
  

• 提前处理弹窗：如果拦截是因为cookie提示、广告弹窗等，在爬虫初始化时直接关闭：

  # 初始化时关闭cookie弹窗
  try:
      cookie_btn = WebDriverWait(driver, 5).until(EC.element_to_be_clickable((By.XPATH, '//button[text()="Accept all"]')))
      cookie_btn.click()
  except:
      pass
  

• 添加数据去重逻辑：存储数据时，以店铺ID、名称等唯一标识为键（比如用字典存储），避免重复数据写入。

3. 数据重复的额外排查点 #

• 检查循环逻辑：是否存在重复遍历同一元素列表的情况（比如翻页后未清空旧列表）？
• 确认翻页逻辑：翻页后是否等待新页面完全加载，再获取新的元素列表？

内容的提问来源于stack exchange，提问作者Geetika Bansal