在AWS Lambda部署Python脚本遇undetected_chromedriver路径异常问题

问题详情 #

使用Docker镜像在AWS Lambda上部署Python自动化脚本，已确认路径配置与安装操作均无误，但运行时出现异常：提示带随机前缀（如e39098076c0be4f2_）的chromedriver需在PATH中，而非常规的chromedriver。更换运行时、chromedriver和发行版版本后问题仍未解决。

完整错误日志 #

{
  "errorMessage": "Message: 'e39098076c0be4f2_chromedriver' executable needs to be in PATH. Please see https://chromedriver.chromium.org/home\n",
  "errorType": "WebDriverException",
  "requestId": "107a17da-65d4-4bec-a5dc-1e69c1b63fe7",
  "stackTrace": [
    "  File \"/home/app/app.py\", line 60, in lambda_handler\n    driver = uc.Chrome(executable_path=r\"/tmp/chromedriver\")\n",
    "  File \"/home/app/undetected_chromedriver/__init__.py\", line 409, in __init__\n    super(Chrome, self).__init__(\n",
    "  File \"/home/app/selenium/webdriver/chrome/webdriver.py\", line 69, in __init__\n    super().__init__(DesiredCapabilities.CHROME['browserName'], \"goog\",\n",
    "  File \"/home/app/selenium/webdriver/chromium/webdriver.py\", line 89, in __init__\n    self.service.start()\n",
    "  File \"/home/app/selenium/webdriver/common/service.py\", line 81, in start\n    raise WebDriverException(\n"
  ]
}

Dockerfile内容 #

# Define global args
ARG FUNCTION_DIR="/home/app/"
ARG RUNTIME_VERSION="3.10"
ARG DISTRO_VERSION="3.16"

# Stage 1 - bundle base image + runtime
# Grab a fresh copy of the image and install GCC
FROM python:${RUNTIME_VERSION}-alpine${DISTRO_VERSION} AS python-alpine
# Install GCC (Alpine uses musl but we compile and link dependencies with GCC)
RUN apk add --no-cache \
    libstdc++

# Stage 2 - build function and dependencies
FROM python-alpine AS build-image
# Install aws-lambda-cpp build dependencies
RUN apk add --no-cache \
    build-base \
    libtool \
    autoconf \
    automake \
    libexecinfo-dev \
    make \
    cmake \
    libcurl \
    curl \
    gcc \
    g++
# Include global args in this stage of the build
ARG FUNCTION_DIR
ARG RUNTIME_VERSION
# Create function directory
RUN mkdir -p ${FUNCTION_DIR}
# Copy required files
COPY patcher.py ${FUNCTION_DIR}
COPY app.py ${FUNCTION_DIR}
COPY requirements.txt .
COPY edit_excutable.py ${FUNCTION_DIR}
# Optional – Install the function's dependencies
RUN python${RUNTIME_VERSION} -m pip install --upgrade pip
RUN python${RUNTIME_VERSION} -m pip install -r requirements.txt --target ${FUNCTION_DIR}
# Fix undetected_chromedriver to use in lambda
RUN cd ${FUNCTION_DIR} && cp -f patcher.py ${FUNCTION_DIR}/undetected_chromedriver
# Install Lambda Runtime Interface Client for Python
RUN python${RUNTIME_VERSION} -m pip install awslambdaric --target ${FUNCTION_DIR}

# Stage 3 - final runtime image
# Grab a fresh copy of the Python image
FROM python-alpine
# Include global arg in this stage of the build
ARG FUNCTION_DIR
# Set working directory to function root directory
WORKDIR ${FUNCTION_DIR}
# Copy in the built dependencies
COPY --from=build-image ${FUNCTION_DIR} ${FUNCTION_DIR}
COPY edit_excutable.py /usr/bin/
RUN apk add chromium-chromedriver
RUN wget https://chromedriver.storage.googleapis.com/107.0.5304.62/chromedriver_linux64.zip

#RUN python${RUNTIME_VERSION} edit_excutable.py
RUN cp /usr/bin/chromedriver ${FUNCTION_DIR}
# (Optional) Add Lambda Runtime Interface Emulator and use a script in the ENTRYPOINT for simpler local runs
ADD https://github.com/aws/aws-lambda-runtime-interface-emulator/releases/latest/download/aws-lambda-rie /usr/bin/aws-lambda-rie
COPY entry.sh /
RUN chmod 755 /usr/bin/aws-lambda-rie /entry.sh
ENTRYPOINT [ "/entry.sh" ]
CMD [ "app.lambda_handler" ]

报错代码片段 #

import os
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support.ui import Select
from selenium.webdriver.support import expected_conditions as EC
import undetected_chromedriver.v2 as uc
import subprocess
import shutil
import time

BIN_DIR = "/tmp/bin"
CURR_BIN_DIR = os.getcwd()

def lambda_handler(event, context):

    if not os.path.exists(BIN_DIR):
        print("Creating bin folder")
        os.makedirs(BIN_DIR)
    os.environ["PATH"] += os.pathsep + BIN_DIR
    os.environ["PATH"] += os.pathsep + CURR_BIN_DIR
    print (os.environ)

    chrome_options = uc.ChromeOptions()
    chrome_options.add_argument('--headless')
    chrome_options.add_argument('--disable-extensions')
    chrome_options.add_argument('--no-sandbox')
    chrome_options.add_argument('--disable-gpu')
    chrome_options.add_argument('--window-size=1024x768')
    chrome_options.add_argument('--user-data-dir=/tmp/user-data')
    chrome_options.add_argument('--hide-scrollbars')
    chrome_options.add_argument('--enable-logging')
    chrome_options.add_argument('--log-level=0')
    chrome_options.add_argument('--single-process')
    chrome_options.add_argument('--data-path=/tmp/data-path')
    chrome_options.add_argument('--ignore-certificate-errors')
    chrome_options.add_argument('--homedir=/tmp')
    chrome_options.add_argument('--disk-cache-dir=/tmp/cache-dir')
    chrome_options.binary_location = "/tmp/chromedriver"
    options = {'request_storage_base_dir': '/tmp' }
    os.system("cp ./chromedriver /tmp/chromedriver")
    os.chmod("/tmp/chromedriver", 0o777)
    driver = uc.Chrome(executable_path=r"/tmp/chromedriver", chrome_options=chrome_options)

解决方案 #

核心原因 #

undetected-chromedriver会自动给chromedriver重命名并添加随机前缀，避免被网站检测，但Lambda环境下，重命名后的文件默认存放在未加入PATH的目录，导致系统找不到该文件。

具体修复步骤 #

1. 修正Chrome二进制路径错误

代码中chrome_options.binary_location = "/tmp/chromedriver"是错误的，该参数需要指向Chrome浏览器的二进制文件，而非chromedriver。在Alpine镜像中，chromium浏览器的路径为/usr/bin/chromium-browser，需修改为：

chrome_options.binary_location = "/usr/bin/chromium-browser"

2. 指定undetected-chromedriver的临时文件目录到PATH包含路径

在初始化driver前，调用uc.install()指定重命名后的chromedriver存放目录为已加入PATH的/tmp/bin，确保系统能找到文件：

# 在复制chromedriver后添加
uc.install(
    driver_executable_path='/tmp/chromedriver',
    browser_executable_path='/usr/bin/chromium-browser',
    target_path=BIN_DIR  # 即/tmp/bin，已加入PATH
)

之后初始化driver时无需再指定executable_path：

driver = uc.Chrome(options=chrome_options)

3. 优化Dockerfile中的chromedriver安装

• 移除重复的wget chromedriver步骤，直接使用apk安装的版本，确保chromium和chromedriver版本一致：

  RUN apk add --no-cache chromium-chromedriver=107.0.5304.87-r0 chromium=107.0.5304.87-r0
  

• 确保chromedriver复制到函数目录，代码中会再复制到/tmp目录，保留该步骤即可。

修正后的代码片段（关键部分） #

def lambda_handler(event, context):

    if not os.path.exists(BIN_DIR):
        print("Creating bin folder")
        os.makedirs(BIN_DIR)
    os.environ["PATH"] += os.pathsep + BIN_DIR
    os.environ["PATH"] += os.pathsep + CURR_BIN_DIR
    print(os.environ)

    chrome_options = uc.ChromeOptions()
    chrome_options.add_argument('--headless')
    chrome_options.add_argument('--disable-extensions')
    chrome_options.add_argument('--no-sandbox')
    chrome_options.add_argument('--disable-gpu')
    chrome_options.add_argument('--window-size=1024x768')
    chrome_options.add_argument('--user-data-dir=/tmp/user-data')
    chrome_options.add_argument('--hide-scrollbars')
    chrome_options.add_argument('--enable-logging')
    chrome_options.add_argument('--log-level=0')
    chrome_options.add_argument('--single-process')
    chrome_options.add_argument('--data-path=/tmp/data-path')
    chrome_options.add_argument('--ignore-certificate-errors')
    chrome_options.add_argument('--homedir=/tmp')
    chrome_options.add_argument('--disk-cache-dir=/tmp/cache-dir')
    # 修正浏览器二进制路径
    chrome_options.binary_location = "/usr/bin/chromium-browser"
    
    # 复制chromedriver到/tmp并设置权限
    os.system("cp ./chromedriver /tmp/chromedriver")
    os.chmod("/tmp/chromedriver", 0o777)
    
    # 指定重命名后的chromedriver存放目录到PATH包含的路径
    uc.install(
        driver_executable_path='/tmp/chromedriver',
        browser_executable_path='/usr/bin/chromium-browser',
        target_path=BIN_DIR
    )
    
    # 初始化driver
    driver = uc.Chrome(options=chrome_options)

内容的提问来源于stack exchange，提问作者EsamHassan