M1 Mac下Selenium ChromeDriver初始化失败测试Dash应用报错

问题表现 #

参照Dash官方测试文档指引，使用selenium Chrome浏览器对Dash应用开展测试，执行测试命令python3 -m pytest location/of/test/folder时触发报错：

• Chrome webdriver超时、无法正确初始化
• chromedriver服务意外退出，状态码为-9
• 最终抛出TestingTimeoutError异常

完整报错堆栈 #

============================================================================================================================== ERRORS ===============================================================================================================================
____________________________________________________________________________________________________________ ERROR at setup of test_bsly001_falsy_child _____________________________________________________________________________________________________________

request = <SubRequest 'dash_duo' for <Function test_bsly001_falsy_child>>, dash_thread_server = <dash.testing.application_runners.ThreadedRunner object at 0x11e57bc70>
tmpdir = local('/private/var/folders/mf/nsy2wm4j33nb995x9rgllwbr0000gn/T/pytest-of-hyrumdickinson/pytest-16/test_bsly001_falsy_child0')

    @pytest.fixture
    def dash_duo(request, dash_thread_server, tmpdir) -> DashComposite:
>       with DashComposite(
            dash_thread_server,
            browser=request.config.getoption("webdriver"),
            remote=request.config.getoption("remote"),
            remote_url=request.config.getoption("remote_url"),
            headless=request.config.getoption("headless"),
            options=request.config.hook.pytest_setup_options(),
            download_path=tmpdir.mkdir("download").strpath,
            percy_assets_root=request.config.getoption("percy_assets"),
            percy_finalize=request.config.getoption("nopercyfinalize"),
            pause=request.config.getoption("pause"),
        ) as dc:

/opt/homebrew/lib/python3.9/site-packages/dash/testing/plugin.py:151: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/opt/homebrew/lib/python3.9/site-packages/dash/testing/composite.py:6: in __init__
    super().__init__(**kwargs)
/opt/homebrew/lib/python3.9/site-packages/dash/testing/browser.py:68: in __init__
    self._driver = until(self.get_webdriver, timeout=1)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

wait_cond = <bound method Browser.get_webdriver of <dash.testing.composite.DashComposite object at 0x11e57b8b0>>, timeout = 1, poll = 0.1, msg = 'expected condition not met within timeout'

    def until(
        wait_cond, timeout, poll=0.1, msg="expected condition not met within timeout"
    ):  # noqa: C0330
        res = wait_cond()
        logger.debug(
            "start wait.until with method, timeout, poll => %s %s %s",
            wait_cond,
            timeout,
            poll,
        )
        end_time = time.time() + timeout
        while not res:
            if time.time() > end_time:
>               raise TestingTimeoutError(msg)
E               dash.testing.errors.TestingTimeoutError: expected condition not met within timeout

/opt/homebrew/lib/python3.9/site-packages/dash/testing/wait.py:26: TestingTimeoutError
------------------------------------------------------------------------------------------------------------------------ Captured log setup -------------------------------------------------------------------------------------------------------------------------
ERROR    dash.testing.browser:browser.py:455 <<<Webdriver not initialized correctly>>>
Traceback (most recent call last):
  File "/opt/homebrew/lib/python3.9/site-packages/dash/testing/browser.py", line 453, in get_webdriver
    return getattr(self, f"_get_{self._browser}")()
  File "/opt/homebrew/lib/python3.9/site-packages/dash/testing/browser.py", line 497, in _get_chrome
    else webdriver.Chrome(options=options)
  File "/opt/homebrew/lib/python3.9/site-packages/selenium/webdriver/chrome/webdriver.py", line 69, in __init__
    super().__init__(DesiredCapabilities.CHROME['browserName'], "goog",
  File "/opt/homebrew/lib/python3.9/site-packages/selenium/webdriver/chromium/webdriver.py", line 89, in __init__
    self.service.start()
  File "/opt/homebrew/lib/python3.9/site-packages/selenium/webdriver/common/service.py", line 98, in start
    self.assert_process_still_running()
  File "/opt/homebrew/lib/python3.9/site-packages/selenium/webdriver/common/service.py", line 110, in assert_process_still_running
    raise WebDriverException(
selenium.common.exceptions.WebDriverException: Message: Service chromedriver unexpectedly exited. Status code was: -9

ERROR    dash.testing.browser:browser.py:455 <<<Webdriver not initialized correctly>>>
Traceback (most recent call last):
  File "/opt/homebrew/lib/python3.9/site-packages/dash/testing/browser.py", line 453, in get_webdriver
    return getattr(self, f"_get_{self._browser}")()
  File "/opt/homebrew/lib/python3.9/site-packages/dash/testing/browser.py", line 497, in _get_chrome
    else webdriver.Chrome(options=options)
  File "/opt/homebrew/lib/python3.9/site-packages/selenium/webdriver/chrome/webdriver.py", line 69, in __init__
    super().__init__(DesiredCapabilities.CHROME['browserName'], "goog",
  File "/opt/homebrew/lib/python3.9/site-packages/selenium/webdriver/chromium/webdriver.py", line 89, in __init__
    self.service.start()
  File "/opt/homebrew/lib/python3.9/site-packages/selenium/webdriver/common/service.py", line 98, in start
    self.assert_process_still_running()
  File "/opt/homebrew/lib/python3.9/site-packages/selenium/webdriver/common/service.py", line 110, in assert_process_still_running
    raise WebDriverException(
selenium.common.exceptions.WebDriverException: Message: Service chromedriver unexpectedly exited. Status code was: -9

ERROR    dash.testing.browser:browser.py:455 <<<Webdriver not initialized correctly>>>
Traceback (most recent call last):
  File "/opt/homebrew/lib/python3.9/site-packages/dash/testing/browser.py", line 453, in get_webdriver
    return getattr(self, f"_get_{self._browser}")()
  File "/opt/homebrew/lib/python3.9/site-packages/dash/testing/browser.py", line 497, in _get_chrome
    else webdriver.Chrome(options=options)
  File "/opt/homebrew/lib/python3.9/site-packages/selenium/webdriver/chrome/webdriver.py", line 69, in __init__
    super().__init__(DesiredCapabilities.CHROME['browserName'], "goog",
  File "/opt/homebrew/lib/python3.9/site-packages/selenium/webdriver/chromium/webdriver.py", line 89, in __init__
    self.service.start()
  File "/opt/homebrew/lib/python3.9/site-packages/selenium/webdriver/common/service.py", line 98, in start
    self.assert_process_still_running()
  File "/opt/homebrew/lib/python3.9/site-packages/selenium/webdriver/common/service.py", line 110, in assert_process_still_running
    raise WebDriverException(
selenium.common.exceptions.WebDriverException: Message: Service chromedriver unexpectedly exited. Status code was: -9
====================================================================================================================== short test summary info ======================================================================================================================
ERROR parmoo/parmoo/viz_test/test_one.py::test_bsly001_falsy_child - dash.testing.errors.TestingTimeoutError: expected condition not met within timeout

测试代码 #

报错时使用的是Dash官方提供的示例测试代码，测试目录下包含空__init__.py文件：

import dash
from dash import html
def test_bsly001_falsy_child(dash_duo):
    app = dash.Dash(__name__)
    app.layout = html.Div(id="nully-wrapper", children=0)
    dash_duo.start_server(app)
    dash_duo.wait_for_text_to_equal("#nully-wrapper", "0", timeout=4)
    assert dash_duo.find_element("#nully-wrapper").text == "0"
    assert dash_duo.get_logs() == [], "browser console should contain no error"
    dash_duo.percy_snapshot("bsly001-layout")

已尝试的排查方案 #

以下操作均未解决问题：

• 将chromedriver加入系统PATH
• 确认chromedriver版本与本地Google Chrome版本匹配
• 确认Google Chrome安装在系统默认路径
• 通过chmod +x [chromedriver路径]命令为chromedriver添加可执行权限
• 清理重复安装的chromedriver
• 删除后重新安装chromedriver
• 通过图形界面与xattr -d com.apple.quarantine /path-to-quarantined-file命令解除chromedriver的系统安全隔离限制
• 确认chromedriver的依赖已全部安装
• 调大timeout参数取值

运行环境 #

• 硬件：2020款M1芯片MacBook Pro
• 系统：macOS Monterey
• 浏览器：Chrome 103
• 驱动：ChromeDriver 103.0.5060.53
• 依赖版本：Python 3.9.13、pytest-7.1.2、dash-2.5.1、selenium-4.3.0、pip-22.1.1、homebrew-3.5.4
• 其他：未使用任何虚拟环境工具

解决步骤 #

1. 校验Chrome与ChromeDriver架构匹配
   M1芯片Mac存在arm64/x86_64转译兼容问题，执行以下两个命令查看二者架构：

file /Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome
file $(which chromedriver)

两个命令的输出必须完全一致，同为arm64或同为x86_64，不匹配则卸载当前chromedriver，安装和Chrome架构、版本完全对应的版本。

2. 修改dash测试插件默认超时配置
   当前dash 2.5.1版本中，webdriver初始化超时被硬编码为1秒，M1芯片下冷启动Chrome耗时经常超过该阈值。打开文件/opt/homebrew/lib/python3.9/site-packages/dash/testing/browser.py，定位到第68行，将self._driver = until(self.get_webdriver, timeout=1)中的timeout=1修改为timeout=10。

3. 显式指定Chrome启动配置
   在测试项目根目录新建conftest.py文件，添加以下配置，绕过默认的路径自动探测逻辑：

import pytest
from selenium.webdriver.chrome.options import Options

@pytest.hookimpl
def pytest_setup_options():
    options = Options()
    options.binary_location = "/Applications/Google Chrome.app/Contents/MacOS/Google Chrome"
    options.add_argument("--headless=new")
    options.add_argument("--no-sandbox")
    options.add_argument("--disable-dev-shm-usage")
    options.add_argument("--disable-gpu")
    return options

4. 降级selenium到兼容版本
   selenium 4.3.0存在Mac ARM架构下的服务启动兼容bug，执行以下命令降级到稳定版本：

pip3 install selenium==4.1.5

完成上述操作后重新执行pytest命令即可正常运行测试。

内容的提问来源于stack exchange，提问作者Hyrum Dickinson