Selenium ChromeDriver：如何避免span元素的StaleElementReferenceException

我正在自动化采集https://bonap.net/TDC/网站的数据，网站加载数据时不会暂停，因此程序在输入搜索查询后需要跟踪显示的搜索结果数量，确保不会在加载完成前采集数据。但获取数值所在的span元素文本时，时常出现StaleElementReferenceException异常。

初始尝试与报错 #

最初代码：

while TimingLoops < 5:
    TimingLoops = TimingLoops+1
    time.sleep(0.05)
    newelement = driver.find_element(By.XPATH, "/html/body/div[3]/div[3]/div[2]/div[1]/div[2]/span[1]")
    newelementText = newelement.text

报错信息：

Exception has occurred: StaleElementReferenceException
Message: stale element reference: stale element not found in the current frame

后续尝试仍报错 #

尝试用WebDriverWait结合EC.presence_of_element_located等待元素出现，代码如下：

while TimingLoops < 5:
    TimingLoops = TimingLoops+1
    time.sleep(0.05)
    newelement = WebDriverWait(driver,120).until(
        EC.presence_of_element_located((By.XPATH, "/html/body/div[3]/div[3]/div[2]/div[1]/div[2]/span[1]"))
    )
    newelementText = newelement.text

报错信息同上。

改用EC.visibility_of_all_elements_located等待，仍会在循环中偶尔触发异常（有时运行20-30次才出现），完整复现代码：

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import time

driver = webdriver.Chrome()
driver.get("https://bonap.net/TDC/")

GenusName = "Rudbeckia"
x=1
GenusCount = 0
wait = WebDriverWait(driver, 10)

while x<100:
    print("The current loop is " + str(x))
    x=x+1
    counts = wait.until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "#genus-panel div.taxon-panel-description > span")))
    ## This is the number of genera before our search
    print("The original number of genera is " + counts[0].text)
    ## Find the Genus Searchbar, delete its current text, and enter our search again
    GenusSearchBar = WebDriverWait(driver,120).until(
        EC.visibility_of_element_located((By.XPATH, "/html/body/div[3]/div[3]/div[2]/div[1]/div[3]/input"))
    )
    GenusSearchBar.clear()
    GenusSearchBar.send_keys(GenusName)
    newcount = wait.until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "#genus-panel div.taxon-panel-description > span")))
    ## This is the number of genera after we search
    print("The new count is " + newcount[0].text)

有效解决方案 #

StaleElementReferenceException本质是元素已被页面重新渲染（DOM更新），原来的元素引用失效。针对该场景，可通过以下方式解决：

1. 每次需要时重新定位元素，避免复用旧引用 #

不要保存元素对象，每次获取文本时重新查找元素。将获取计数的逻辑封装成函数：

def get_genus_count(wait):
    # 每次调用都重新等待并定位元素
    count_element = wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, "#genus-panel div.taxon-panel-description > span")))
    return count_element.text

循环中使用示例：

while x<100:
    print("The current loop is " + str(x))
    x=x+1
    # 直接调用函数获取最新计数
    original_count = get_genus_count(wait)
    print("The original number of genera is " + original_count)
    
    GenusSearchBar = wait.until(EC.visibility_of_element_located((By.XPATH, "/html/body/div[3]/div[3]/div[2]/div[1]/div[3]/input")))
    GenusSearchBar.clear()
    GenusSearchBar.send_keys(GenusName)
    
    # 再次调用函数获取更新后的计数
    new_count = get_genus_count(wait)
    print("The new count is " + new_count)

2. 等待DOM稳定，确认元素文本不再变化 #

网站搜索后计数会动态更新，可等待计数文本停止变化，确保页面加载完成：

def get_stable_genus_count(wait, timeout=10):
    start_time = time.time()
    previous_count = None
    while time.time() - start_time < timeout:
        current_count = wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, "#genus-panel div.taxon-panel-description > span"))).text
        if current_count == previous_count:
            return current_count
        previous_count = current_count
        time.sleep(0.1)
    # 超时返回最后获取的计数
    return previous_count

3. 捕获异常并重试 #

在获取元素的逻辑中捕获StaleElementReferenceException，失败时重试：

from selenium.common.exceptions import StaleElementReferenceException

def get_genus_count_with_retry(wait, max_retries=3):
    for _ in range(max_retries):
        try:
            count_element = wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, "#genus-panel div.taxon-panel-description > span")))
            return count_element.text
        except StaleElementReferenceException:
            # 捕获异常后继续重试
            continue
    # 多次重试失败后抛出异常或返回默认值
    raise Exception("Failed to get genus count after multiple retries")

4. 优化定位方式，避免绝对XPATH #

绝对XPATH（如/html/body/div[3]/div[3]/...）非常脆弱，页面结构稍有变化就会失效，且更容易受DOM更新影响。继续使用你已经在使用的相对CSS选择器#genus-panel div.taxon-panel-description > span即可，这种定位方式更稳定。

内容的提问来源于stack exchange，提问作者Samantha Clark