使用Selenium间歇性出现JavascriptException异常求助

问题描述 #

执行Selenium代码中的waiting.until(EC.visibility_of_element_located((By.CLASS_NAME, "csgofloat-itemfloat")))时，间歇性抛出错误：

Exception has occurred: JavascriptException / Message: javascript error: this.each is not a function

代码有时正常运行，有时报错，完整代码如下：

from selenium import webdriver
import time
import requests
from cgitb import text
from unittest import result
from bs4 import BeautifulSoup
from selenium.webdriver.common.by import By
from selenium.common.exceptions import NoSuchElementException
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

driver.get("https://steamcommunity.com/market/listings/730/M4A4%20%7C%20Mainframe%20%28Minimal%20Wear%29")

waiting = WebDriverWait(driver, 10)
waiting.until(EC.visibility_of_element_located((By.CLASS_NAME, "csgofloat-itemfloat")))

page_source = driver.page_source
soup = BeautifulSoup(page_source, "html.parser")
hopefully_floats = soup.find_all("div", {"class": "csgofloat-itemfloat"})

time.sleep(15)
print(hopefully_floats)
print(type(hopefully_floats))
driver.quit()

解决思路 #

1. 定位问题根源 #

这个错误是页面JS冲突导致的：this.each是jQuery方法，报错说明某个对象并非jQuery集合，大概率是CSGOFloat这类第三方注入脚本还没完成初始化，就被Selenium的可见性检查触发了异常。

2. 替换等待条件规避JS操作 #

把visibility_of_element_located换成presence_of_element_located，后者只检查元素是否存在于DOM中，不会触发元素可见性相关的JS逻辑，能避开冲突：

waiting.until(EC.presence_of_element_located((By.CLASS_NAME, "csgofloat-itemfloat")))

3. 先等页面完全加载 #

在等待目标元素前，先确保页面DOM和脚本都加载完成：

# 等待页面加载完成（document ready状态）
waiting.until(lambda driver: driver.execute_script('return document.readyState') == 'complete')

# 再等待目标元素
waiting.until(EC.presence_of_element_located((By.CLASS_NAME, "csgofloat-itemfloat")))

4. 加异常重试机制 #

针对间歇性错误，给等待逻辑加重试：

from selenium.common.exceptions import JavascriptException

max_retries = 3
retry_count = 0
while retry_count < max_retries:
    try:
        waiting.until(EC.visibility_of_element_located((By.CLASS_NAME, "csgofloat-itemfloat")))
        break
    except JavascriptException:
        retry_count += 1
        time.sleep(2)  # 重试前等待2秒
else:
    raise RuntimeError("多次重试后仍无法获取目标元素")

5. 禁用浏览器扩展 #

如果是CSGOFloat浏览器插件引发的冲突，初始化driver时禁用扩展：

options = webdriver.ChromeOptions()
options.add_argument('--disable-extensions')
driver = webdriver.Chrome(options=options)

内容的提问来源于stack exchange，提问作者jetrldz