Selenium多会话超32个时Chrome崩溃无法创建会话求助

我们尝试用Selenium同时处理大量页面，最多能正常运行32个会话，但当会话数量进一步增加时，所有会话开始出现卡顿、报错甚至崩溃，还频繁触发无法创建会话的错误。

已测试5个以上版本的Selenium（覆盖v102至最新v114），搭配对应版本的ChromeDriver和Chrome，问题依旧，Chrome持续崩溃。也尝试过hub-node架构和独立运行模式，均无改善。

确认内存不是瓶颈，已通过-Xms64g -Xmx80g调整JVM堆内存，且按要求安装了Oracle JDK（而非JRE），但问题仍未解决。如何实现运行超过32个会话？

环境信息 #

操作系统 #

Ubuntu 22.04.2 LTS，32核Ryzen™ 9处理器，128GB内存

Java版本 #

java version "20.0.1" 2023-04-18
Java(TM) SE Runtime Environment (build 20.0.1+9-29)
Java HotSpot(TM) 64-Bit Server VM (build 20.0.1+9-29, mixed mode, sharing)

Google Chrome版本 #

114.0.5735.133

Selenium启动命令 #

xvfb-run -a java -Dwebdriver.chrome.driver="/home/forge/domain/storage/selenium/chromedriver-114.0.5735.16" -jar "/home/forge/domain/storage/selenium/selenium-server-4.10.0.jar" standalone --override-max-sessions true --max-sessions 60 --port 4646 --allow-cors true

内存统计 #

Mem:       131576260    15530844    76271352       82296    39774064   114669600
Swap:        5237752           0     5237752

会话创建方式 #

RemoteWebDriver::create($host, $capabilities,3600000,3600000);

Chrome启动参数 #

'--remote-allow-origins=*',
'--window-size=1920,1080',
'--enable-experimental-extension-apis',
'--disable-infobars',
'--disable-notifications',
'--disable-device-discovery-notifications',
'--disable-gpu',
'--allow-outdated-plugins',
'--disable-web-security',
'--allow-running-insecure-content',
'--allow-file-access-from-log_files',
'--audio-output-channels=0',
'--ignore-certificate-error',
'--disable-dev-shm-usage',
'--disable-browser-side-navigation',
'--dns-prefetch-disable',
'--disable-geolocation',
'--disable-search-geolocation-disclosure',
'--enable-strict-powerful-feature-restrictions',
'--disable-translate',
'--disable-sync',
'--mute-audio',
'--lang=en-US',
'--use-fake-ui-for-media-stream',
'--use-fake-device-for-media-stream',
'--allow-file-access-from-files',
'--reduce-security-for-testing',
'--allow-insecure-localhost',
'--allow-legacy-extension-manifests',
'--allow-loopback-in-peer-connection',
'--allow-outdated-plugins',
'--allow-unsecure-dlls',
'--alsa-mute-device-name',
'--also-emit-success-logs',
'--alsa-mute-element-name',
'--no-first-run',
'--disable-default-apps',
'--disable-extensions-http-throttling',
'--disable-blink-features=AutomationControlled',

错误日志 #

10:39:23.636 WARN [SeleniumSpanExporter$1.lambda$export$1] - Unable to create session: Could not start a new session. Error while creating session with the driver service. Stopping driver service: Could not start a new session. Response code 500. Message: unknown error: 
Chrome failed to start: exited abnormally.
  (unknown error: DevToolsActivePort file doesn't exist)
  (The process started from chrome location /usr/bin/google-chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.) 
Host info: host: 'domain', ip: '127.0.0.1'
Build info: version: '4.10.0', revision: 'c14d967899'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-75-generic', java.version: '20.0.1'
Driver info: driver.version: unknown
Build info: version: '4.10.0', revision: 'c14d967899'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-75-generic', java.version: '20.0.1'
Driver info: driver.version: unknown
10:39:23.636 WARN [SeleniumSpanExporter$1.lambda$export$1] - org.openqa.selenium.SessionNotCreatedException: Could not start a new session. Error while creating session with the driver service. Stopping driver service: Could not start a new session. Response code 500. 
Message: unknown error: Chrome failed to start: exited abnormally.
  (unknown error: DevToolsActivePort file doesn't exist)
  (The process started from chrome location /usr/bin/google-chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.) 
Host info: host: 'domain', ip: '127.0.0.1'
Build info: version: '4.10.0', revision: 'c14d967899'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-75-generic', java.version: '20.0.1'
Driver info: driver.version: unknown
Build info: version: '4.10.0', revision: 'c14d967899'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.15.0-75-generic', java.version: '20.0.1'
Driver info: driver.version: unknown
    at org.openqa.selenium.grid.node.config.DriverServiceSessionFactory.apply(DriverServiceSessionFactory.java:210)
    at org.openqa.selenium.grid.node.config.DriverServiceSessionFactory.apply(DriverServiceSessionFactory.java:69)
    at org.openqa.selenium.grid.node.local.SessionSlot.apply(SessionSlot.java:147)
    at org.openqa.selenium.grid.node.local.LocalNode.newSession(LocalNode.java:467)
    at org.openqa.selenium.grid.distributor.local.LocalDistributor.startSession(LocalDistributor.java:643)
    at org.openqa.selenium.grid.distributor.local.LocalDistributor.newSession(LocalDistributor.java:560)
    at org.openqa.selenium.grid.distributor.local.LocalDistributor$NewSessionRunnable.handleNewSessionRequest(LocalDistributor.java:814)
    at org.openqa.selenium.grid.distributor.local.LocalDistributor$NewSessionRunnable.lambda$run$1(LocalDistributor.java:773)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
    at java.base/java.lang.Thread.run(Thread.java:1623)

解决方案 #

1. 优化Chrome启动参数 #

清理冗余参数（如重复的--allow-outdated-plugins、Linux无效的--allow-unsecure-dlls），并添加资源优化参数：

'--headless=new',          # 启用新版无头模式，大幅节省资源
'--single-process',        # 强制单进程运行，减少进程开销
'--disable-background-networking',
'--disable-background-timer-throttling',
'--disable-backgrounding-occluded-windows',
'--disable-renderer-backgrounding',
'--no-sandbox'             # 关闭沙箱（仅测试环境使用）

2. 调整XVFB配置 #

为XVFB指定足够显存，避免显示资源不足：

xvfb-run -a -s "-screen 0 1920x1080x24 -ac -nolisten tcp" java -Dwebdriver.chrome.driver="/home/forge/domain/storage/selenium/chromedriver-114.0.5735.16" -jar "/home/forge/domain/storage/selenium/selenium-server-4.10.0.jar" standalone --override-max-sessions true --max-sessions 60 --port 4646 --allow-cors true

3. 提升系统进程与文件句柄限制 #

编辑/etc/security/limits.conf添加：

* soft nofile 65535
* hard nofile 65535
* soft nproc 65535
* hard nproc 65535

临时生效执行：

sysctl -w kernel.pid_max=65535

永久生效需编辑/etc/sysctl.conf添加kernel.pid_max=65535，再执行sysctl -p。

4. 优化Selenium Grid线程池 #

添加线程池参数，匹配最大会话数：

xvfb-run -a java -Xms64g -Xmx80g -Dwebdriver.chrome.driver="/home/forge/domain/storage/selenium/chromedriver-114.0.5735.16" -jar "/home/forge/domain/storage/selenium/selenium-server-4.10.0.jar" standalone --override-max-sessions true --max-sessions 60 --port 4646 --allow-cors true --session-timeout 3600 --node-threads 60

内容的提问来源于stack exchange，提问作者Guntram