在Airflow(Docker)任务中初始化ChromeDriver遇权限拒绝错误[Errno 13]

核心原因 #

undetected_chromedriver会自动修改ChromeDriver可执行文件（添加补丁绕过反爬检测），但你指定的/usr/local/bin/chromedriver位于系统目录，Airflow运行用户（默认是airflow）没有该文件的写入权限，因此触发PermissionError。

有效解决方案 #

以下是几种可行的长期解决方式，按优先级排序：

1. 将ChromeDriver复制到Airflow用户可写入的目录
   在初始化脚本中，先把系统目录的chromedriver复制到Airflow的工作目录（比如/opt/airflow/drivers/），再指定该路径给undetected_chromedriver：

   import os
   import shutil
   from undetected_chromedriver import Chrome as uc
   
   def initialize_driver(headless=True):
       # 定义目标路径，确保airflow用户有读写权限
       target_driver_path = '/opt/airflow/drivers/chromedriver'
       source_driver_path = '/usr/local/bin/chromedriver'
       
       # 确保目录存在并复制文件
       os.makedirs(os.path.dirname(target_driver_path), exist_ok=True)
       if not os.path.exists(target_driver_path):
           shutil.copy(source_driver_path, target_driver_path)
           os.chmod(target_driver_path, 0o755)  # 添加执行权限
       
       driver = uc.Chrome(
           headless=headless,
           executable_path=target_driver_path
       )
       return driver
   

2. 启用子进程模式避免修改原文件
   给undetected_chromedriver传入use_subprocess=True参数，通过子进程启动ChromeDriver，绕过对原文件的修改操作：

   from undetected_chromedriver import Chrome as uc
   
   def initialize_driver(headless=True):
       driver = uc.Chrome(
           headless=headless,
           executable_path='/usr/local/bin/chromedriver',
           use_subprocess=True
       )
       return driver
   

3. Docker镜像构建阶段预补丁ChromeDriver
   在构建Airflow Docker镜像时，提前用root用户完成ChromeDriver的补丁操作，容器启动后无需再修改文件：

   RUN python -c "from undetected_chromedriver import Patcher; p = Patcher(); p.patch_exe()"
   

   注意：执行完补丁命令后，记得切换回airflow用户继续构建镜像。

已尝试步骤回顾 #

你已完成以下排查动作，但未触及核心的写入权限问题：

• 检查并添加chromedriver的执行权限
• 验证PATH环境变量配置
• 重新安装与Chrome兼容的ChromeDriver版本
• 临时用sudo测试权限
• 更新Selenium和undetected_chromedriver依赖

错误细节 #

错误日志

[2025-01-18, 22:30:22 UTC] {logging_mixin.py:190} INFO - Failed to initialize WebDriver: [Errno 13] Permission denied: '/usr/local/bin/chromedriver'

出错代码

from undetected_chromedriver import Chrome as uc

def initialize_driver(headless=True):
    driver = uc.Chrome(
        headless=headless,
        executable_path='/usr/local/bin/chromedriver'
    )
    return driver

完整回溯

Traceback (most recent call last):
  File "/opt/airflow/dags/job_scraper_dag.py", line 40, in scrape_and_store
    driver = initialize_driver(headless=config["HEADLESS"])
  File "/opt/airflow/dags/app/scrape.py", line 54, in initialize_driver
    driver = uc.Chrome()
  File "/home/airflow/.local/lib/python3.11/site-packages/undetected_chromedriver/__init__.py", line 258, in __init__
    self.patcher.auto()
  File "/home/airflow/.local/lib/python3.11/site-packages/undetected_chromedriver/patcher.py", line 150, in auto
    return self.patch_exe()
  File "/home/airflow/.local/lib/python3.11/site-packages/undetected_chromedriver/patcher.py", line 347, in patch_exe
    with io.open(self.executable_path, "r+b") as fh:
PermissionError: [Errno 13] Permission denied: '/usr/local/bin/chromedriver'

环境信息

• Airflow版本：2.10.4
• Python版本：3.11
• ChromeDriver版本：132.0.6834.83

内容的提问来源于stack exchange，提问作者Qi Xiang