无头服务器SSH运行脚本时undetected-chromedriver无法连接Chrome求助

在Ubuntu无头服务器通过SSH运行Python脚本，此前运行正常，近期做了无关小改动后，undetected-chromedriver出现连接失败问题。直接使用Selenium无头模式无异常。

运行环境 #

• 操作系统：Ubuntu 16.04.7 LTS
• 内核：Linux 4.4.0-210-generic
• 架构：x86-64
• Chrome版本：Google Chrome 103.0.5060.134
• Python版本：3.7.7
• 依赖版本：undetected-chromedriver==3.1.5.post4

简化代码 #

from undetected_chromedriver import find_chrome_executable
from selenium.webdriver.common.by import By
import undetected_chromedriver as uc
import time
from logger import logger


def scrape():
    try:
        print(find_chrome_executable())
        driver = uc.Chrome(headless=True, service_args=["--verbose", "--log-path=cd.log"])

        driver.get('https://myexternalip.com/raw')
        time.sleep(1)
        print(driver.find_elements(By.CSS_SELECTOR, "body")[0].text)
    except Exception as e:
        print(e)
        logger.exception("Exception occured: ")
scrape()

错误输出 #

标准输出/错误 #

/usr/bin/google-chrome-stable
Message: unknown error: cannot connect to chrome at 127.0.0.1:37090
from chrome not reachable
Stacktrace:
#0 0x55ee5283acd3 <unknown>
#1 0x55ee526427ff <unknown>
#2 0x55ee52631bc9 <unknown>
#3 0x55ee5266a69f <unknown>
#4 0x55ee526625a6 <unknown>
#5 0x55ee5269d94a <unknown>
#6 0x55ee52697aa3 <unknown>
#7 0x55ee5266d3fa <unknown>
#8 0x55ee5266e555 <unknown>
#9 0x55ee528822bd <unknown>
#10 0x55ee52886418 <unknown>
#11 0x55ee5286c36e <unknown>
#12 0x55ee52887078 <unknown>
#13 0x55ee52860bb0 <unknown>
#14 0x55ee528a3d58 <unknown>
#15 0x55ee528a3ed8 <unknown>
#16 0x55ee528bdcfd <unknown>
#17 0x7ff1fe2e76ba <unknown>

完整堆栈跟踪 #

patching driver executable /home/kpsingh/.local/share/undetected_chromedriver/fcf5cf8f5e8b441d_chromedriver
Exception occured: 
Traceback (most recent call last):
  File "a.py", line 12, in scrape
    driver = uc.Chrome(options=opts, headless=True, service_args=["--verbose", "--log-path=cd.log"])
  File "/data-2/research/expedia/Ranking-Comparator/venv/lib/python3.7/site-packages/undetected_chromedriver/__init__.py", line 408, in __init__
    keep_alive=keep_alive,
  File "/data-2/research/expedia/Ranking-Comparator/venv/lib/python3.7/site-packages/selenium/webdriver/chrome/webdriver.py", line 72, in __init__
    service_log_path, service, keep_alive)
  File "/data-2/research/expedia/Ranking-Comparator/venv/lib/python3.7/site-packages/selenium/webdriver/chromium/webdriver.py", line 97, in __init__
    options=options)
  File "/data-2/research/expedia/Ranking-Comparator/venv/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 277, in __init__
    self.start_session(capabilities, browser_profile)
  File "/data-2/research/expedia/Ranking-Comparator/venv/lib/python3.7/site-packages/undetected_chromedriver/__init__.py", line 590, in start_session
    capabilities, browser_profile
  File "/data-2/research/expedia/Ranking-Comparator/venv/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 370, in start_session
    response = self.execute(Command.NEW_SESSION, parameters)
  File "/data-2/research/expedia/Ranking-Comparator/venv/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 435, in execute
    self.error_handler.check_response(response)
  File "/data-2/research/expedia/Ranking-Comparator/venv/lib/python3.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 247, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: cannot connect to chrome at 127.0.0.1:37090
from chrome not reachable
Stacktrace:
#0 0x55ee5283acd3 <unknown>
#1 0x55ee526427ff <unknown>
#2 0x55ee52631bc9 <unknown>
#3 0x55ee5266a69f <unknown>
#4 0x55ee526625a6 <unknown>
#5 0x55ee5269d94a <unknown>
#6 0x55ee52697aa3 <unknown>
#7 0x55ee5266d3fa <unknown>
#8 0x55ee5266e555 <unknown>
#9 0x55ee528822bd <unknown>
#10 0x55ee52886418 <unknown>
#11 0x55ee5286c36e <unknown>
#12 0x55ee52887078 <unknown>
#13 0x55ee52860bb0 <unknown>
#14 0x55ee528a3d58 <unknown>
#15 0x55ee528a3ed8 <unknown>
#16 0x55ee528bdcfd <unknown>
#17 0x7ff1fe2e76ba <unknown>

Driver详细日志 #

[1658781393.276][INFO]: Starting ChromeDriver 103.0.5060.134 (8ec6fce403b3feb0869b0732eda8bd95011d333c-refs/branch-heads/5060@{#1262}) on port 47324
[1658781393.276][INFO]: Please see https://chromedriver.chromium.org/security-considerations for suggestions on keeping ChromeDriver safe.
[1658781393.779][INFO]: [c0a694c101f2727524e75d070370b885] COMMAND InitSession {
   "capabilities": {
      "alwaysMatch": {
         "browserName": "chrome",
         "goog:chromeOptions": {
            "args": [ "--remote-debugging-host=127.0.0.1", "--remote-debugging-port=37090", "--user-data-dir=/tmp/tmp2mvwlfbl", "--lang=en-US", "--no-default-browser-check", "--no-first-run", "--headless", "--window-size=1920,1080", "--start-maximized", "--no-sandbox", "--log-level=0" ],
            "binary": "/usr/bin/google-chrome-stable",
            "debuggerAddress": "127.0.0.1:37090",
            "extensions": [  ]
         },
         "pageLoadStrategy": "normal"
      },
      "firstMatch": [ {
      } ]
   }
}
[1658781393.779][DEBUG]: DevTools HTTP Request: http://127.0.0.1:37090/json/version
[1658781393.781][DEBUG]: DevTools HTTP Request failed
[1658781393.831][DEBUG]: DevTools HTTP Request: http://127.0.0.1:37090/json/version
[1658781393.833][DEBUG]: DevTools HTTP Request failed
[1658781393.883][DEBUG]: DevTools HTTP Request: http://127.0.0.1:37090/json/version
[1658781393.884][DEBUG]: DevTools HTTP Request failed
[1658781393.934][DEBUG]: DevTools HTTP Request: http://127.0.0.1:37090/json/version
[1658781393.935][DEBUG]: DevTools HTTP Request failed
[1658781393.985][DEBUG]: DevTools HTTP Request: http://127.0.0.1:37090/json/version
[1658781393.986][DEBUG]: DevTools HTTP Request failed
.
.
.
[1658781453.688][DEBUG]: DevTools HTTP Request: http://127.0.0.1:37090/json/version
[1658781453.689][DEBUG]: DevTools HTTP Request failed
[1658781453.739][DEBUG]: DevTools HTTP Request: http://127.0.0.1:37090/json/version
[1658781453.740][DEBUG]: DevTools HTTP Request failed
[1658781453.790][DEBUG]: DevTools HTTP Request: http://127.0.0.1:37090/json/version
[1658781453.791][DEBUG]: DevTools HTTP Request failed
[1658781453.796][INFO]: [c0a694c101f2727524e75d070370b885] RESPONSE InitSession ERROR unknown error: cannot connect to chrome at 127.0.0.1:37090
from chrome not reachable
[1658781453.796][DEBUG]: Log type 'driver' lost 0 entries on destruction
[1658781453.796][DEBUG]: Log type 'browser' lost 0 entries on destruction

解决方案 #

1. 指定Chrome二进制路径 #

undetected-chromedriver自动识别的Chrome路径与系统实际路径不一致，手动指定二进制文件路径避免冲突：

opts = uc.ChromeOptions()
opts.binary_location = "/opt/google/chrome/chrome"
driver = uc.Chrome(options=opts, headless=True, service_args=["--verbose", "--log-path=cd.log"])

2. 补充无头环境必要启动参数 #

添加以下参数解决无头环境下Chrome启动异常问题：

opts.add_argument("--disable-dev-shm-usage")  # 解决/dev/shm空间不足问题
opts.add_argument("--disable-gpu")  # 无头模式下禁用GPU加速
opts.add_argument("--remote-debugging-port=0")  # 让系统自动分配端口，避免端口占用冲突

3. 更新undetected-chromedriver版本 #

当前使用的版本较旧，更新到适配Chrome 103的兼容版本：

pip install --upgrade undetected-chromedriver==3.5.3

4. 清除旧driver缓存 #

删除undetected-chromedriver生成的缓存文件，重新生成适配当前Chrome版本的driver：

rm -rf /home/kpsingh/.local/share/undetected_chromedriver/

5. 验证Chrome手动启动权限 #

确保运行脚本的用户有足够权限启动Chrome，执行以下命令验证：

/opt/google/chrome/chrome --headless --no-sandbox --disable-gpu https://myexternalip.com/raw

内容的提问来源于stack exchange，提问作者Parth Singh