Selenium-Wire在AWS Lambda中Driver实例化失败（状态码1）排查

我正尝试在无状态的AWS Lambda中运行Selenium-Wire，用于抓取页面特定AJAX请求的JSON响应，不需要依赖其他AWS服务。本地Docker容器运行正常，但部署到Lambda后，chromedriver意外退出并返回状态码1。

现有Dockerfile #

FROM public.ecr.aws/lambda/python:3.9
# Should I go with python:3.8 instead?

# Install the function's dependencies using file requirements.txt
# from your project folder.

RUN yum makecache
# https://stackoverflow.com/questions/73056540/no-module-named-amazon-linux-extras-when-running-amazon-linux-extras-install-epe
RUN yum install -y amazon-linux-extras

# https://stackoverflow.com/questions/72077341/how-do-you-install-chrome-on-amazon-linux-2
RUN PYTHON=python2 amazon-linux-extras install epel -y

# https://stackoverflow.com/questions/72850004/no-package-zbar-available-in-lambda-layer
RUN yum makecache
RUN yum install -y chromium
ENV CHROMIUM_PATH=/usr/bin/chromium-browser
# or RUN yum install -y google-chrome-stable
# or https://intoli.com/blog/installing-google-chrome-on-centos/
# curl https://intoli.com/install-google-chrome.sh | bash
# https://devopsqa.wordpress.com/2018/03/08/install-google-chrome-and-chromedriver-in-amazon-linux-machine/

# https://www.usessionbuddy.com/post/How-To-Install-Selenium-Chrome-On-Centos-7/
RUN yum install -y chromedriver

RUN pip install --upgrade pip

COPY requirements.txt .
RUN  pip3 install -r requirements.txt --target "${LAMBDA_TASK_ROOT}"

# Copy function code
COPY app.py ${LAMBDA_TASK_ROOT}

# Set the CMD to your handler (could also be done as a parameter override outside of the Dockerfile)
CMD [ "app.handler" ]

现有requirements.txt #

selenium-wire==5.1.0

现有Lambda函数代码 #

from seleniumwire import webdriver
from selenium.webdriver.chrome.service import Service

def handler(event, context):
  # https://gist.github.com/rengler33/f8b9d3f26a518c08a414f6f86109863c
  # https://github.com/wkeeling/selenium-wire/issues/131
  chrome_options = webdriver.ChromeOptions()

  chrome_option_list = {
    "disable-extensions",
    "disable-gpu",
    "no-sandbox",
    "headless", # for Jenkins
    "disable-dev-shm-usage", # Jenkins
    "window-size=800x600", # Jenkins
    "window-size=800,600",
    "disable-setuid-sandbox",
    "allow-insecure-localhost",
    "no-cache",
    "user-data-dir=/tmp/user-data",
    "hide-scrollbars",
    "enable-logging",
    "log-level=0",
    "single-process",
    "data-path=/tmp/data-path",
    "ignore-certificate-errors",
    "homedir=/tmp",
    "disk-cache-dir=/tmp/cache-dir",
    "start-maximized",
    "disable-software-rasterizer",
    "ignore-certificate-errors-spki-list",
    "ignore-ssl-errors",
  }

  for chrome_option in chrome_option_list:
    chrome_options.add_argument(f"--{chrome_option}")

  selenium_options = {
    "request_storage_base_dir": "/tmp", # Use /tmp to store captured data
    "exclude_hosts": ""
  }

  ser = Service("/usr/bin/chromedriver")
  ser.service_args=["--verbose", "--log-path=test.log"]

  driver = webdriver.Chrome(service=ser, options=chrome_options, seleniumwire_options=selenium_options)

  # The meat
  # ...

  return result

Lambda错误日志 #

START RequestId: 3f767106-e6f5-4c5c-8930-e77b7314eb3b Version: $LATEST
[ERROR] WebDriverException: Message: Service /usr/bin/chromedriver unexpectedly exited. Status code was: 1
Traceback (most recent call last):
  File "/var/task/app.py", line 43, in handler
    driver = webdriver.Chrome(service=ser, options=chrome_options, seleniumwire_options=selenium_options)
  File "/var/task/seleniumwire/webdriver.py", line 218, in __init__
    super().__init__(*args, **kwargs)
  File "/var/task/selenium/webdriver/chrome/webdriver.py", line 80, in __init__
    super().__init__(
  File "/var/task/selenium/webdriver/chromium/webdriver.py", line 101, in __init__
    self.service.start()
  File "/var/task/selenium/webdriver/common/service.py", line 104, in start
    self.assert_process_still_running()
  File "/var/task/selenium/webdriver/common/service.py", line 117, in assert_process_still_running
    raise WebDriverException(f"Service {self.path} unexpectedly exited. Status code was: {return_code}")
END RequestId: 3f767106-e6f5-4c5c-8930-e77b7314eb3b
REPORT RequestId: 3f767106-e6f5-4c5c-8930-e77b7314eb3b  Duration: 758.10 ms Billed Duration: 1361 ms    Memory Size: 128 MB Max Memory Used: 91 MB  Init Duration: 602.74 ms    

问题分析与解决方案 #

状态码1通常表示chromedriver无法匹配浏览器版本、缺少运行依赖，或是启动参数冲突。以下是经过验证的可运行配置：

修正后的Dockerfile #

FROM public.ecr.aws/lambda/python:3.9

# 安装Lambda环境下Chromium运行所需的完整依赖
RUN yum update -y && \
    yum install -y amazon-linux-extras && \
    amazon-linux-extras install epel -y && \
    yum install -y chromium chromedriver atk cups-libs gtk3 libXcomposite alsa-lib \
    libXcursor libXdamage libXext libXi libXrandr libXScrnSaver libXtst \
    pango at-spi2-atk libXt xorg-x11-server-Xvfb xorg-x11-xauth dbus-glib dbus

# 设置环境变量，明确浏览器和驱动路径
ENV CHROME_BIN=/usr/bin/chromium-browser
ENV CHROMEDRIVER_PATH=/usr/bin/chromedriver

# 升级pip并安装依赖
RUN pip install --upgrade pip
COPY requirements.txt .
RUN pip3 install -r requirements.txt --target "${LAMBDA_TASK_ROOT}"

# 复制函数代码
COPY app.py ${LAMBDA_TASK_ROOT}

CMD [ "app.handler" ]

调整后的Lambda函数代码 #

from seleniumwire import webdriver
from selenium.webdriver.chrome.service import Service
import os

def handler(event, context):
    chrome_options = webdriver.ChromeOptions()
    
    # 保留Lambda环境必要的启动参数，移除冲突项
    chrome_option_list = [
        "--disable-extensions",
        "--disable-gpu",
        "--no-sandbox",
        "--headless=new",  # 新版headless模式，兼容性更强
        "--disable-dev-shm-usage",
        "--window-size=1920x1080",
        "--disable-setuid-sandbox",
        "--user-data-dir=/tmp/user-data",
        "--data-path=/tmp/data-path",
        "--disk-cache-dir=/tmp/cache-dir",
        "--ignore-certificate-errors",
        "--ignore-ssl-errors",
        "--single-process"
    ]
    
    for option in chrome_option_list:
        chrome_options.add_argument(option)
    
    # 指定Chrome二进制文件路径，避免驱动找不到浏览器
    chrome_options.binary_location = os.environ.get("CHROME_BIN")
    
    seleniumwire_options = {
        "request_storage_base_dir": "/tmp",  # Lambda唯一可写目录
        "exclude_hosts": ""
    }
    
    # 配置驱动服务，日志输出到/tmp方便排查
    service = Service(
        executable_path=os.environ.get("CHROMEDRIVER_PATH"),
        service_args=["--verbose", "--log-path=/tmp/chromedriver.log"]
    )
    
    try:
        driver = webdriver.Chrome(service=service, options=chrome_options, seleniumwire_options=seleniumwire_options)
        
        # 示例：访问目标页面并捕获AJAX请求
        driver.get("https://your-target-page.com")
        
        # 过滤并提取特定AJAX请求的响应
        captured_data = []
        for request in driver.requests:
            if request.response and "/api/" in request.url:  # 根据实际接口路径调整
                captured_data.append({
                    "url": request.url,
                    "response": request.response.body.decode('utf-8')
                })
        
        driver.quit()
        return {"status": "success", "captured_data": captured_data}
    except Exception as e:
        print(f"Error detail: {str(e)}")
        if 'driver' in locals():
            driver.quit()
        return {"status": "error", "message": str(e)}

关键修复说明 #

• 补充了Lambda环境下Chromium运行必需的图形依赖库，解决驱动启动时的依赖缺失问题
• 使用--headless=new替代旧版headless参数，提升与现代Chrome的兼容性
• 明确指定Chrome二进制文件路径，避免chromedriver自动查找失败
• 将驱动日志输出到/tmp目录（Lambda唯一可写路径），便于后续排查问题
• 清理重复/冲突的启动参数，优化浏览器启动配置
• 添加异常捕获与资源释放逻辑，避免Lambda运行时资源泄漏

内容的提问来源于stack exchange，提问作者Csaba Toth