Selenium无头Chrome模式下按钮交互异常问题求助

问题概述 #

编写的Selenium代码在正常Chrome模式下可完成谷歌搜索、点击首个结果、点击页面按钮的操作，但切换到无头Chrome模式后，执行到点击按钮步骤时抛出IndexError: list index out of range。测试谷歌、YouTube、ChatGPT、TikTok等站点均出现该问题，仅example.com可正常运行。尝试使用undetected_chromedriver未解决，添加--headless=new参数消除了Cloudflare相关错误，但索引问题仍存在。

正常模式可运行代码 #

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait

browser = webdriver.Chrome()
browser.get("https://www.google.com/search?q=chatgpt")
f = browser.find_elements(By.TAG_NAME, "h3")
f[0].click()
button = browser.find_elements(By.TAG_NAME, 'button')
button[0].click()
print("Page title was '{}'".format(browser.title))
input()

正常模式输出 #

DevTools listening on ws://127.0.0.1:58358/devtools/browser/e8f33ba5-9423-4900-9d36-035615599b61
Page title was 'ChatGPT'
[6464:11928:0914/044721.426:ERROR:device_event_log_impl.cc(225)] [04:47:21.426] USB: usb_service_win.cc:415 Could not read device interface GUIDs: The system cannot find the file specified. (0x2)

无头模式无法运行代码 #

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait

chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("--no-sandbox")
chrome_options.add_argument("--headless")
chrome_options.add_argument("--disable-gpu")
browser = webdriver.Chrome(options=chrome_options)
browser.get("https://www.google.com/search?q=chatgpt")
f = browser.find_elements(By.TAG_NAME, "h3")
f[0].click()
button = browser.find_elements(By.TAG_NAME, 'button')
button[0].click()
print("Page title was '{}'".format(browser.title))
input()

无头模式报错信息 #

DevTools listening on ws://127.0.0.1:58289/devtools/browser/e4c33772-2e8c-4060-96b6-6aa730ef53c2
[0914/044642.023:INFO:CONSOLE(0)] "Error with Permissions-Policy header: Origin trial controlled feature not enabled: 'unload'.", source:  (0)
[0914/044644.747:INFO:CONSOLE(0)] "Error with Permissions-Policy header: Origin trial controlled feature not enabled: 'unload'.", source:  (0)
[0914/044645.283:INFO:CONSOLE(0)] "Error with Permissions-Policy header: Origin trial controlled feature not enabled: 'browsing-topics'.", source:  (0)
[0914/044645.284:INFO:CONSOLE(0)] "Error with Permissions-Policy header: Origin trial controlled feature not enabled: 'interest-cohort'.", source:  (0)
[0914/044645.435:INFO:CONSOLE(0)] "Refused to execute script from 'https://chat.openai.com/cdn-cgi/challenge-platform/h/g/scripts/alpha/invisible.js?ts=1694649600' because its MIME type ('') is not executable, and strict MIME type checking is enabled.", source: about:blank (0)
[0914/044646.033:INFO:CONSOLE(0)] "Error with Permissions-Policy header: Origin trial controlled feature not enabled: 'browsing-topics'.", source:  (0)
[0914/044646.033:INFO:CONSOLE(0)] "Error with Permissions-Policy header: Origin trial controlled feature not enabled: 'interest-cohort'.", source:  (0)
Traceback (most recent call last):
  File "d:\RapidAPI\willitworkorwillittwerk.py", line 14, in <module>
    button[0].click()
    ~~~~~~^^^
IndexError: list index out of range

版本信息 #

• Chrome版本：117.0.5938.63
• Webdriver版本：117.0.5938.62
• Selenium版本：4.12.0

解决方案 #

1. 设置窗口尺寸 #

无头模式默认窗口尺寸极小，会导致页面元素布局异常，部分元素未渲染。添加窗口尺寸参数模拟正常浏览器窗口：

chrome_options.add_argument("--window-size=1920,1080")

2. 显式等待元素加载 #

避免页面未完全加载就执行操作，使用WebDriverWait等待目标元素可见或可点击，确保元素已存在于DOM中：

# 等待首个h3元素可点击
first_result = WebDriverWait(browser, 10).until(
    EC.element_to_be_clickable((By.TAG_NAME, "h3"))
)
first_result.click()

# 等待首个button元素可点击
target_button = WebDriverWait(browser, 10).until(
    EC.element_to_be_clickable((By.TAG_NAME, "button"))
)
target_button.click()

3. 模拟正常User-Agent #

无头模式的默认User-Agent与正常浏览器不同，部分网站会返回差异化内容。添加匹配当前Chrome版本的User-Agent：

chrome_options.add_argument("user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36")

修改后的完整代码 #

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait

chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("--no-sandbox")
chrome_options.add_argument("--headless=new")
chrome_options.add_argument("--disable-gpu")
chrome_options.add_argument("--window-size=1920,1080")
chrome_options.add_argument("user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36")

browser = webdriver.Chrome(options=chrome_options)
browser.get("https://www.google.com/search?q=chatgpt")

# 等待并点击首个搜索结果
first_result = WebDriverWait(browser, 10).until(
    EC.element_to_be_clickable((By.TAG_NAME, "h3"))
)
first_result.click()

# 等待并点击首个按钮
target_button = WebDriverWait(browser, 10).until(
    EC.element_to_be_clickable((By.TAG_NAME, "button"))
)
target_button.click()

print("Page title was '{}'".format(browser.title))
browser.quit()

内容的提问来源于stack exchange，提问作者Caiden