设置后仍无法运行Brave无头模式，出现NoSuchElementException错误

问题描述 #

无头模式运行Selenium脚本时，始终报NoSuchElementException找不到settings-ui元素，但非无头模式完全正常，错误信息如下：

selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":"settings-ui"}

(Session info: headless chrome=116.1.57.62)

环境信息 #

• 浏览器：Brave
• Selenium版本：4.11.2
• Chromium内核版本：116.0.5845.180

核心问题原因 #

1. 旧无头模式行为差异：你用的--headless是旧版无头模式，和正常浏览器的渲染、加载逻辑差异极大，Brave的设置页面在这种模式下加载不完整。
2. 无等待逻辑：直接链式查找多层Shadow DOM元素，完全没考虑页面加载时间，无头模式下页面渲染速度本来就慢，元素还没出来就执行查找了。
3. 配置冗余/错误：比如start-maximized和window-size冲突，User-Agent格式带多余引号，还有已经过时的disable-gpu参数。

具体修复步骤 #

1. 切换到新版无头模式 #

Chrome 112+推出了--headless=new参数，这个模式和正常浏览器行为几乎一致，替换掉原来的--headless：

options.add_argument("--headless=new")

2. 增加显式等待 #

用WebDriverWait等待settings-ui元素出现，给足页面加载时间，最长建议设10-15秒：

# 先导入需要的模块
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

# 在driver.get之后添加等待逻辑
settings_ui = WebDriverWait(driver, 15).until(
    EC.presence_of_element_located((By.CSS_SELECTOR, "settings-ui"))
)

3. 清理冗余配置 #

• 移除--start-maximized，和--window-size同时设置会冲突，保留后者即可
• 移除-disable-gpu，新版无头模式已经不需要这个参数
• 修正User-Agent格式，不要加额外引号，同时换成Brave专属的UA：

user_agent = 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36 Brave/1.56.14'
options.add_argument(f'user-agent={user_agent}')

4. 修改后的完整代码 #

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import Select, WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.chrome.service import Service

options = Options()
options.binary_location = '/snap/brave/current/opt/brave.com/brave/brave'
options.add_argument("--user-data-dir=/home/dhruv/snap/brave/current/.config/BraveSoftware/Brave-Browser/")
options.add_argument("--profile-directory=Default")
options.add_argument("--disable-extensions")
options.add_argument("--proxy-server='direct://'")
options.add_argument("--proxy-bypass-list=*")
options.add_argument("--disable-blink-features=AutomationControlled")
options.add_argument("--disable-web-security")
options.add_argument("--disable-xss-auditor")
options.add_argument("--allow-running-insecure-content")
options.add_argument("--ignore-certificate-errors")
options.add_argument("--headless=new")  # 新版无头模式
options.add_argument("--remote-debugging-port=9222")
options.add_argument("--window-size=1366,768")
options.add_experimental_option("detach", True)

# 使用Brave专属User-Agent
user_agent = 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36 Brave/1.56.14'
options.add_argument(f'user-agent={user_agent}')

driver_path = '/home/dhruv/Downloads/chromedriver-linux64/chromedriver'
service = Service(executable_path=driver_path, log_path = '/home/dhruv/python/chromedriver.log')
driver = webdriver.Chrome(options = options, service = service)

try :
    driver.get("brave://settings/appearance")
    driver.get_screenshot_as_file("screenshot.png")
    
    # 等待settings-ui元素加载完成
    settings_ui = WebDriverWait(driver, 15).until(
        EC.presence_of_element_located((By.CSS_SELECTOR, "settings-ui"))
    )
    
    elm = settings_ui.shadow_root\
            .find_element(By.CSS_SELECTOR, "settings-main").shadow_root\
            .find_element(By.CSS_SELECTOR, "settings-basic-page").shadow_root\
            .find_element(By.CSS_SELECTOR, "[section=appearance] settings-appearance-page").shadow_root\
            .find_element(By.CSS_SELECTOR, "settings-animated-pages settings-brave-appearance-theme").shadow_root\
            .find_element(By.CSS_SELECTOR, "#braveThemeType")
    select = Select(elm)
    if select.first_selected_option.get_attribute("value") == "1":
        select.select_by_value("2")
    else:
        select.select_by_value("1")
finally :
    driver.close()
    driver.quit()

内容的提问来源于stack exchange，提问作者Dhruv