Upwork高级搜索输入触发Selenium StaleElementReferenceException求助

报错信息 #

selenium.common.exceptions.StaleElementReferenceException: Message: stale element reference: element is not attached to the page document
  (Session info: chrome=108.0.5359.95)

此前尝试过现有解决方案，但因Selenium代码更新均无效。

相关页面 #

Upwork高级搜索弹窗页面（对应链接：https://www.upwork.com/nx/jobs/search/modals/advanced-search?sort=recency&pageTitle=Advanced%20Search&_navType=modal&_modalInfo=%5B%7B%22navType%22%3A%22modal%22,%22title%22%3A%22Advanced%20Search%22,%22modalId%22%3A%221670133126002%22,%22channelName%22%3A%22advanced-search-modal%22%7D%5D）

原始代码 #

import time

from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

options = Options()
options.add_argument("start-maximized")

webdriver_service = Service('F:\\work\\chromedriver_win32\\chromedriver.exe')
driver = webdriver.Chrome(options=options, service=webdriver_service)
wait = WebDriverWait(driver, 10)

url = "https://www.upwork.com/nx/jobs/search/?sort=recency"
driver.get(url)

key = ["Web Scraping","Selenium WebDriver", "Data Scraping", "selenium", "Web Crawling", "Beautiful Soup", "Scrapy", "Data Extraction", "Automation"]
wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, 'button#onetrust-accept-btn-handler')))
time.sleep(5)
wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, 'button#onetrust-accept-btn-handler'))).click()
for i in range(len(key)):
    wait.until(EC.element_to_be_clickable((By.XPATH, '//button[contains(@title,"Advanced Search")]'))).click()
    time.sleep(5)
    advanced_search_input = driver.find_element(By.XPATH,'//input[contains(@aria-labelledby,"tokenizer-label")]')
    # advanced_search_input.click()
    advanced_search_input.send_keys(key[i])

解决方案 #

报错核心原因是：每次打开高级搜索弹窗后，页面DOM会重新渲染，直接使用driver.find_element会获取到已失效的元素引用。修改思路是每次操作都通过显式等待获取最新的元素，替换不可靠的time.sleep。

修改后的代码 #

from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

options = Options()
options.add_argument("start-maximized")

webdriver_service = Service('F:\\work\\chromedriver_win32\\chromedriver.exe')
driver = webdriver.Chrome(options=options, service=webdriver_service)
wait = WebDriverWait(driver, 15)

url = "https://www.upwork.com/nx/jobs/search/?sort=recency"
driver.get(url)

# 处理Cookie弹窗
wait.until(EC.element_to_be_clickable((By.ID, 'onetrust-accept-btn-handler'))).click()

keywords = ["Web Scraping","Selenium WebDriver", "Data Scraping", "selenium", "Web Crawling", "Beautiful Soup", "Scrapy", "Data Extraction", "Automation"]

for keyword in keywords:
    # 打开高级搜索弹窗
    wait.until(EC.element_to_be_clickable((By.XPATH, '//button[contains(@title,"Advanced Search")]'))).click()
    
    # 等待技能搜索输入框加载完成并可交互
    search_input = wait.until(EC.visibility_of_element_located((By.XPATH, '//input[contains(@aria-labelledby,"tokenizer-label")]')))
    # 清空输入框（防止残留内容）
    search_input.clear()
    # 输入关键词
    search_input.send_keys(keyword)
    
    # 可根据需求添加后续操作：比如点击搜索按钮或关闭弹窗
    # wait.until(EC.element_to_be_clickable((By.XPATH, '//button[contains(text(),"Search")]'))).click()
    # wait.until(EC.element_to_be_clickable((By.XPATH, '//button[contains(@aria-label,"Close")]'))).click()

关键修改点 #

• 移除所有time.sleep，改用WebDriverWait显式等待，确保元素完全加载后再操作
• 每次循环打开弹窗后，重新等待输入框可见，获取最新的DOM元素引用，彻底避免 stale element 问题
• 添加search_input.clear()，防止输入框残留之前的内容
• 延长等待超时时间到15秒，适配Upwork页面的加载速度
• 简化Cookie弹窗处理逻辑，直接点击无需重复等待

内容的提问来源于stack exchange，提问作者Info Rewind