Docker容器中无法运行Selenium脚本，请求技术排查

问题 #

尝试在Docker容器中运行Selenium脚本时出现以下错误：

Traceback (most recent call last):
2023-12-12 15:44:32   File "/usr/local/lib/python3.12/site-packages/django/core/handlers/exception.py", line 55, in inner
2023-12-12 15:44:32     response = get_response(request)
2023-12-12 15:44:32                ^^^^^^^^^^^^^^^^^^^^^
2023-12-12 15:44:32   File "/usr/local/lib/python3.12/site-packages/django/core/handlers/base.py", line 197, in _get_response
2023-12-12 15:44:32     response = wrapped_callback(request, *callback_args, **callback_kwargs)
2023-12-12 15:44:32                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-12-12 15:44:32   File "/code/fyp_project/web_scraper/views.py", line 9, in user_interface
2023-12-12 15:44:32     scrape_reviews(product_url)
2023-12-12 15:44:32   File "/code/fyp_project/web_scraper/helpers.py", line 34, in scrape_reviews
2023-12-12 15:44:32     browser = webdriver.Chrome(service=ChromeService(ChromeDriverManager().install()))
2023-12-12 15:44:32               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-12-12 15:44:32   File "/usr/local/lib/python3.12/site-packages/selenium/webdriver/chrome/webdriver.py", line 45, in __init__
2023-12-12 15:44:32     super().__init__(
2023-12-12 15:44:32   File "/usr/local/lib/python3.12/site-packages/selenium/webdriver/chromium/webdriver.py", line 61, in __init__
2023-12-12 15:44:32     super().__init__(command_executor=executor, options=options)
2023-12-12 15:44:32   File "/usr/local/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 209, in __init__
2023-12-12 15:44:32     self.start_session(capabilities)
2023-12-12 15:44:32   File "/usr/local/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 293, in start_session
2023-12-12 15:44:32     response = self.execute(Command.NEW_SESSION, caps)["value"]
2023-12-12 15:44:32                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-12-12 15:44:32   File "/usr/local/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 348, in execute
2023-12-12 15:44:32     self.error_handler.check_response(response)
2023-12-12 15:44:32   File "/usr/local/lib/python3.12/site-packages/selenium/webdriver/remote/errorhandler.py", line 229, in check_response
2023-12-12 15:44:32     raise exception_class(message, screen, stacktrace)
2023-12-12 15:44:32 selenium.common.exceptions.SessionNotCreatedException: Message: session not created: Chrome failed to start: exited normally.
2023-12-12 15:44:32   (session not created: DevToolsActivePort file doesn't exist)
2023-12-12 15:44:32   (The process started from chrome location /opt/google/chrome/chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.)

已修改Python代码添加启动参数，但错误依旧：

def scrape_reviews(product_url):
    chrome_path='/usr/bin/google-chrome'
    filename='daraz_reviews.csv'
    #open the browser
    #service = Service(executable_path=path)
    options = webdriver.ChromeOptions()
    options.binary_location=chrome_path
    options.add_argument("--disable-gpu")
    options.add_argument("--disable-extensions")
    options.add_argument("--disable-infobars")
    options.add_argument("--start-maximized")
    options.add_argument("--disable-notifications")
    options.add_argument('--headless')
    options.add_argument('--no-sandbox')
    options.add_argument('--disable-dev-shm-usage')
    options.add_argument("--disable-setuid-sandbox") 
    browser = webdriver.Chrome(service=ChromeService(ChromeDriverManager().install())) #Error at this line

当前使用的Dockerfile：

# Use an official Python runtime as a parent image
FROM python:3.12

# Allows docker to cache installed dependencies between builds
COPY requirements.txt requirements.txt
RUN pip install --no-cache-dir -r requirements.txt

# Install dependencies for Chrome and Chromedriver
RUN apt-get update && apt-get install -y \
    wget \
    unzip \
    libnss3 \
    && rm -rf /var/lib/apt/lists/*

#Download Google Chrome dependencies
# Chrome dependency Instalation
RUN apt-get update && apt-get install -y \
    fonts-liberation \
    libasound2 \
    libatk-bridge2.0-0 \
    libatk1.0-0 \
    libatspi2.0-0 \
    libcups2 \
    libdbus-1-3 \
    libdrm2 \
    libgbm1 \
    libgtk-3-0 \
    libgtk-4-1 \
    libnspr4 \
    libnss3 \
    libwayland-client0 \
    libxcomposite1 \
    libxdamage1 \
    libxfixes3 \
    libxkbcommon0 \
    libxrandr2 \
    xdg-utils \
    libu2f-udev \
    libvulkan1

#Download Google Chrome
RUN apt -f install -y
RUN apt-get install -y wget
RUN wget -q https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
RUN dpkg -i ./google-chrome-stable_current_amd64.deb 

#Display the path to google chrome
RUN which google-chrome

# Download and install Chromedriver
RUN wget -O /tmp/chromedriver.zip https://edgedl.me.gvt1.com/edgedl/chrome/chrome-for-testing/120.0.6099.71/linux64/chromedriver-linux64.zip && \
   unzip /tmp/chromedriver.zip -d /usr/local/bin/ && \
   rm /tmp/chromedriver.zip


# Set the PATH environment variable
ENV PATH="/usr/local/bin:${PATH}"

# Mount the application code to the image
COPY . /code/fyp_project/
WORKDIR /code

EXPOSE 8000

# Run the production server
ENTRYPOINT ["python", "fyp_project/manage.py"]
CMD ["runserver", "0.0.0.0:8000"]

解决方案 #

1. 修复Python代码的参数传递问题 #

你创建了ChromeOptions对象但未传给webdriver.Chrome，这是核心问题。同时优化部分参数适配无头模式：

def scrape_reviews(product_url):
    chrome_path='/usr/bin/google-chrome'
    filename='daraz_reviews.csv'
    options = webdriver.ChromeOptions()
    options.binary_location=chrome_path
    options.add_argument("--disable-gpu")
    options.add_argument("--disable-extensions")
    options.add_argument("--disable-infobars")
    # 替换start-maximized为固定窗口大小，避免无头模式兼容性问题
    options.add_argument("--window-size=1920,1080")
    options.add_argument("--disable-notifications")
    # Chrome 112+推荐使用新无头模式格式
    options.add_argument('--headless=new')
    options.add_argument('--no-sandbox')
    options.add_argument('--disable-dev-shm-usage')
    options.add_argument("--disable-setuid-sandbox") 
    # 将配置好的options传递给Chrome驱动
    browser = webdriver.Chrome(options=options, service=ChromeService(ChromeDriverManager().install()))

2. 对齐Chrome与ChromeDriver版本 #

Dockerfile中手动下载了固定版本的ChromeDriver，但安装的是最新稳定版Chrome，易出现版本不匹配问题。两种解决方式：

• 方式一：依赖ChromeDriverManager自动匹配
  删除Dockerfile中手动下载ChromeDriver的所有步骤，让ChromeDriverManager自动安装对应版本的驱动。
• 方式二：手动匹配版本
  修改Dockerfile中ChromeDriver下载逻辑，自动获取已安装Chrome的版本并下载对应驱动：

  # 下载并安装与Chrome版本匹配的ChromeDriver
  RUN CHROME_VERSION=$(google-chrome --version | grep -oP '\d+\.\d+\.\d+\.\d+') && \
      wget -O /tmp/chromedriver.zip https://edgedl.me.gvt1.com/edgedl/chrome/chrome-for-testing/$CHROME_VERSION/linux64/chromedriver-linux64.zip && \
      unzip /tmp/chromedriver.zip -d /usr/local/bin/ && \
      rm /tmp/chromedriver.zip
  

3. 优化Dockerfile的依赖安装 #

合并重复命令，减少镜像层数并清理冗余文件：

#Download Google Chrome
RUN apt-get install -y wget && \
    wget -q https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb && \
    dpkg -i ./google-chrome-stable_current_amd64.deb && \
    apt-get -f install -y && \
    rm ./google-chrome-stable_current_amd64.deb && \
    rm -rf /var/lib/apt/lists/*

4. 验证Chrome可执行路径 #

确保Dockerfile中RUN which google-chrome的输出与代码中chrome_path一致，若输出不是/usr/bin/google-chrome，则调整代码中的路径。

内容的提问来源于stack exchange，提问作者Muhammad Taimoor