在Amazon Lightsail的Flask服务中Selenium WebDriver启动失败问题

本地运行正常的Flask爬虫服务，部署到Amazon Lightsail的Docker容器后，点击网页按钮启动爬虫时Chrome无法正常启动，容器本身运行状态正常，疑似Chrome WebDriver版本不匹配或Docker环境缺失必要配置导致。

错误日志 #

super().__init__(
File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/chromium/webdriver.py", line 61, in __init__
super().__init__(command_executor=executor, options=options)
File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 208, in __init__
self.start_session(capabilities)
File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 292, in start_session
response = self.execute(Command.NEW_SESSION, caps)["value"]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 347, in execute
self.error_handler.check_response(response)
File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/remote/errorhandler.py", line 229, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.SessionNotCreatedException: Message: session not created: Chrome failed to start: exited abnormally.
(timeout: Timed out receiving message from renderer: 60.000)
(The process started from chrome location /opt/google/chrome/chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
Stacktrace:

现有Dockerfile #

# Set base image (host OS)
FROM debian:latest

# Install necessary dependencies
RUN apt-get update -y \
    && apt-get upgrade -y \
    && apt-get install -y \
        wget \
        gnupg \
        unzip \
        xz-utils \
        build-essential \
        zlib1g-dev \
        libncurses5-dev \
        libgdbm-dev \
        libssl-dev \
        libreadline-dev \
        libffi-dev \
        libsqlite3-dev \
        libbz2-dev \
        libnss3-dev \
        openssl \
    && export DEBIAN_FRONTEND=noninteractive

# Install Chrome
RUN wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - \
    && echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google-chrome.list \
    && apt-get update && apt-get install -y \
    google-chrome-stable

# Install Chrome WebDriver - seems to get installed with Chrome
#RUN wget -q -O /usr/local/bin/chromedriver https://chromedriver.storage.googleapis.com/114.0.5735.90/chromedriver_linux64.zip \
#    && unzip /usr/local/bin/chromedriver -d /usr/local/bin \
#    && chmod +x /usr/local/bin/chromedriver

# Install Python 3.11.3
RUN wget -O /tmp/python.tar.xz https://www.python.org/ftp/python/3.11.3/Python-3.11.3.tar.xz \
    && mkdir -p /usr/src/python \
    && tar -xJC /usr/src/python --strip-components=1 -f /tmp/python.tar.xz \
    && rm /tmp/python.tar.xz \
    && cd /usr/src/python \
    && ./configure --enable-optimizations --with-ssl \
    && make -j$(nproc) \
    && make install \
    && cd / \
    && rm -rf /usr/src/python \
    && python3 -c "import ssl; print(ssl.OPENSSL_VERSION)"

# Update pip
RUN pip3 install --upgrade pip

# Clean up
RUN rm -rf /var/lib/apt/lists/*

# Copy the dependencies file to the working directory
COPY requirements.txt /tmp/

# Install python dependencies
RUN pip3 install -r /tmp/requirements.txt

requirements.txt #

flask===2.2.2
matplotlib===3.8.0
numpy===1.24.3
pandas===1.5.3
pytz===2023.3
selenium===4.18.1
webdriver_manager===4.0.1
werkzeug===2.2.2
waitress===3.0.0
google-auth-oauthlib===1.1.0
google-api-python-client===2.122.0
undetected_chromedriver===3.5.5

解决方案 #

1. 添加Chrome无头模式及Docker专用启动参数 #

Docker容器无图形界面，必须启用无头模式并关闭沙箱，否则Chrome无法启动。在初始化Driver时添加以下参数：

from selenium.webdriver.chrome.options import Options

chrome_options = Options()
# 无头模式（Selenium 4.10+推荐使用new参数）
chrome_options.add_argument("--headless=new")
# 禁用沙箱（Docker环境必须，否则Chrome会因权限问题崩溃）
chrome_options.add_argument("--no-sandbox")
# 禁用/dev/shm共享内存，避免容器内存不足导致崩溃
chrome_options.add_argument("--disable-dev-shm-usage")
# 固定窗口大小，避免无头模式下渲染异常
chrome_options.add_argument("--window-size=1920,1080")
# 禁用GPU（无头模式不需要）
chrome_options.add_argument("--disable-gpu")

# 普通Selenium Driver初始化
from selenium.webdriver.chrome.service import Service
from selenium.webdriver import Chrome
driver = Chrome(service=Service(), options=chrome_options)

# 若使用undetected-chromedriver
import undetected_chromedriver as uc
driver = uc.Chrome(options=chrome_options)

2. 确保Chrome与ChromeDriver版本严格匹配 #

• 取消Dockerfile中ChromeDriver安装的注释，先查看容器内Chrome版本（google-chrome-stable --version），下载对应版本的chromedriver替换原有链接；
• 或使用webdriver-manager自动管理版本，确保代码中正确调用：

  from webdriver_manager.chrome import ChromeDriverManager
  driver = Chrome(service=Service(ChromeDriverManager().install()), options=chrome_options)
  

3. 补充Docker容器中缺失的Chrome依赖 #

在Dockerfile的apt-get install列表中添加以下依赖，解决Chrome运行时的库缺失问题：

xvfb \
libxi6 \
libgconf-2-4 \
fonts-liberation \

4. 避免以root用户运行Chrome #

Chrome默认拒绝root用户启动，建议在Dockerfile末尾添加普通用户创建步骤：

# 创建普通用户并切换
RUN useradd -m appuser
USER appuser

5. 优化Chrome安装方式 #

替换原Dockerfile中的Chrome安装步骤为官方deb包安装，避免源同步问题：

# 替换原Chrome安装代码
RUN wget -q https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
RUN apt-get install -y ./google-chrome-stable_current_amd64.deb
RUN rm google-chrome-stable_current_amd64.deb

内容的提问来源于stack exchange，提问作者adamruehle