Selenium远程WebDriver使用Chrome配置文件报错:无法创建默认配置目录
当前使用ChromeDriver v102版本(v103版本存在已登记公开Bug,故未升级)。
在Selenium Grid节点上远程运行ChromeDriver时,抛出如下错误:
Response code 500. Message: unknown error: cannot create default profile directory
- 本地运行ChromeDriver时,指定Chrome配置文件可正常加载使用
- Firefox驱动远程运行无异常,可判定该问题为ChromeDriver远程运行场景专属问题
操作流程与已做排查
操作逻辑为从云端下载Chrome配置文件到当前工作目录,代码可正常识别定位该目录,路径无空格,配置文件内仅保留核心必要文件,保留列表如下:
save_file_list = ["Bookmarks", "History", "History-journal", "Cookies", "Cookies-Journal", "Favicons", "Favicons-journal"]
本地运行ChromeDriver时,驱动可自动加载上述核心文件,自主生成其余所需文件,相同代码在Windows、Mac设备运行(含打包为可执行程序场景)均正常。
远程运行场景下已完成如下排查:
- 不指定
user-data-dir参数时远程驱动运行完全正常,认证代理配置可正常工作,排除网络连接问题 - 虚拟环境运行代码、Docker镜像运行均会复现错误,排除Docker环境导致问题的可能
相关实现代码
加载Chrome配置文件到Chrome选项、初始化远程驱动的代码如下:
def add_profile(self, chrome_options): if self.download_profile(): arg = f'--user-data-dir={os.getcwd()}/{self.profile_path}' logging.info(f'Adding arg: {arg}') chrome_options.add_argument(arg) logging.info("Successfully added profile to chrome driver.") return chrome_options def setup_chrome(self) -> webdriver.Remote: prefs = { "credentials_enable_service": False, "profile.password_manager_enabled": False, } chrome_options = webdriver.ChromeOptions() chrome_options.add_argument("--enable-javascript") chrome_options.add_argument("--disable-blink-features=AutomationControlled") chrome_options.add_experimental_option("useAutomationExtension", False) chrome_options.add_experimental_option("excludeSwitches", ["enable-automation"]) chrome_options.add_experimental_option("prefs", prefs) chrome_options = self.proxy.add_proxy_plugin(chrome_options, use_proxy=True) chrome_options = self.chrome_profile.add_profile(chrome_options) # 该配置当前远程场景下不生效 capabilities = { "browserName": "chrome", "chromeOptions": { "useAutomationExtension": False, "forceDevToolsScreenshot": True, "args": ["--start-maximized", "--disable-infobars"], }, } remote_driver_host = os.environ.get("SELENIUM_URL") logging.info("Setting up remote connection") try: selenium_connection = RemoteConnectionV2( remote_driver_host, keep_alive=False ) selenium_connection.set_remote_connection_authentication_headers() driver = webdriver.Remote( command_executor=selenium_connection, # 生产环境地址 # command_executor="http://localhost:4444/wd/hub", # 本地开发调试地址 desired_capabilities=capabilities, options=chrome_options, keep_alive=True, ) logging.info("SUCCESS: created remote chrome using connection") except Exception as ex: logging.error(f"Failed to initiate Chrome browser: {ex}") raise try: driver.get("https://www.google.com") except Exception as e: logging.error(f"Failed to connect using proxy: {e}") return driver
运行错误日志
27-Jun-22 10:04:53 AM EDT | INFO | SUCCESS: downloaded proxy assignment: 27-Jun-22 10:04:54 AM EDT | INFO | Successfully downloaded & extracted _zGF_23_CHROME_PROFILE. 27-Jun-22 10:04:54 AM EDT | INFO | Adding arg: --user-data-dir=/Users/<myuser>/Repos<my-repo>/_zGF_23_CHROME_PROFILE 27-Jun-22 10:04:54 AM EDT | INFO | Successfully added profile to chrome driver. 27-Jun-22 10:04:54 AM EDT | INFO | Setting up remote connection [Authentication] No identity token was found in the environment. Requesting a new one. [Authentication] An identity token found successfully. 27-Jun-22 10:04:55 AM EDT | ERROR | Failed to initiate Chrome browser: Message: Could not start a new session. Error while creating session with the driver service. Stopping driver service: Could not start a new session. Response code 500. Message: unknown error: cannot create default profile directory Build info: version: '4.2.2', revision: '683ccb65d6' System info: host: 'localhost', ip: '127.0.0.1', os.name: 'Linux', os.arch: 'amd64', os.version: '4.4.0', java.version: '11.0.15' Driver info: driver.version: unknown Build info: version: '4.2.2', revision: '683ccb65d6' System info: host: 'localhost', ip: '127.0.0.1', os.name: 'Linux', os.arch: 'amd64', os.version: '4.4.0', java.version: '11.0.15' Driver info: driver.version: unknown Stacktrace: at org.openqa.selenium.grid.node.config.DriverServiceSessionFactory.apply (DriverServiceSessionFactory.java:194) at org.openqa.selenium.grid.node.config.DriverServiceSessionFactory.apply (DriverServiceSessionFactory.java:67) at org.openqa.selenium.grid.node.local.SessionSlot.apply (SessionSlot.java:145) at org.openqa.selenium.grid.node.local.LocalNode.newSession (LocalNode.java:362) at org.openqa.selenium.grid.distributor.local.LocalDistributor.startSession (LocalDistributor.java:618) at org.openqa.selenium.grid.distributor.local.LocalDistributor.newSession (LocalDistributor.java:544) at org.openqa.selenium.grid.distributor.local.LocalDistributor$NewSessionRunnable.handleNewSessionRequest (LocalDistributor.java:791) at org.openqa.selenium.grid.distributor.local.LocalDistributor$NewSessionRunnable.lambda$run$1 (LocalDistributor.java:752) at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1128) at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:628) at java.lang.Thread.run (Thread.java:829) 27-Jun-22 10:04:55 AM EDT | ERROR | Message: Could not start a new session. Error while creating session with the driver service. Stopping driver service: Could not start a new session. Response code 500. Message: unknown error: cannot create default profile directory Build info: version: '4.2.2', revision: '683ccb65d6' System info: host: 'localhost', ip: '127.0.0.1', os.name: 'Linux', os.arch: 'amd64', os.version: '4.4.0', java.version: '11.0.15' Driver info: driver.version: unknown Build info: version: '4.2.2', revision: '683ccb65d6' System info: host: 'localhost', ip: '127.0.0.1', os.name: 'Linux', os.arch: 'amd64', os.version: '4.4.0', java.version: '11.0.15' Driver info: driver.version: unknown Stacktrace: at org.openqa.selenium.grid.node.config.DriverServiceSessionFactory.apply (DriverServiceSessionFactory.java:194) at org.openqa.selenium.grid.node.config.DriverServiceSessionFactory.apply (DriverServiceSessionFactory.java:67) at org.openqa.selenium.grid.node.local.SessionSlot.apply (SessionSlot.java:145) at org.openqa.selenium.grid.node.local.LocalNode.newSession (LocalNode.java:362) at org.openqa.selenium.grid.distributor.local.LocalDistributor.startSession (LocalDistributor.java:618) at org.openqa.selenium.grid.distributor.local.LocalDistributor.newSession (LocalDistributor.java:544) at org.openqa.selenium.grid.distributor.local.LocalDistributor$NewSessionRunnable.handleNewSessionRequest (LocalDistributor.java:791) at org.openqa.selenium.grid.distributor.local.LocalDistributor$NewSessionRunnable.lambda$run$1 (LocalDistributor.java:752) at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1128) at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:628) at java.lang.Thread.run (Thread.java:829)
该错误的核心原因是本地路径与Selenium Grid节点文件系统不互通:
本地运行ChromeDriver时,--user-data-dir指向的是执行代码的本机路径,驱动可以直接访问;但远程Selenium Grid运行ChromeDriver时,驱动进程实际运行在Grid节点的Linux环境中,代码里传入的是本地设备的路径(日志显示路径为/Users/<myuser>/Repos<my-repo>/_zGF_23_CHROME_PROFILE),Grid节点的文件系统里不存在这个路径,Chrome启动时既找不到对应目录,也没有权限在节点对应路径下创建默认配置文件夹,因此抛出错误。
可通过以下两种方案修复:
- 文件挂载方案:将Chrome配置文件提前存放到所有Selenium Grid节点的固定路径下,启动节点时将该路径挂载为容器可读写目录,代码中
--user-data-dir参数传入Grid节点内的绝对路径,而非本地代码运行机的路径。 - 远程文件传输方案:初始化驱动前,先通过接口将本地的Chrome配置文件压缩包上传到Grid节点的临时可写目录,解压后再将节点内的解压路径传入
--user-data-dir参数,会话结束后自动清理节点上的临时配置文件。
额外注意:代码中同时传了desired_capabilities和chrome_options,Selenium 4.x版本中desired_capabilities的部分配置会和options冲突,建议把所有Chrome启动参数全部合并到ChromeOptions中,移除冗余的capabilities配置,避免参数被覆盖。
内容的提问来源于stack exchange,提问作者Iris

