Docker环境下Python+Selenium服务运行一段时间后崩溃求助

我开发了一个基于Python 3.10和Selenium 4.10.0的网页数据解析服务，在Docker环境中运行时，会在5-15分钟后崩溃，抛出如下异常：

Traceback (most recent call last):
app       |   File "//app.py", line 199, in <module>
app       |     driver.get(link)
app       |   File "/usr/local/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 355, in get
app       |     self.execute(Command.GET, {"url": url})
app       |   File "/usr/local/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 346, in execute
app       |     self.error_handler.check_response(response)
app       |   File "/usr/local/lib/python3.10/site-packages/selenium/webdriver/remote/errorhandler.py", line 245, in check_response
app       |     raise exception_class(message, screen, stacktrace)
app       | selenium.common.exceptions.InvalidSessionIdException: Message: invalid session id
app       | Stacktrace:
app       | #0 0x55ac01c904e3 <unknown>
app       | #1 0x55ac019bfb00 <unknown>
app       | #2 0x55ac019ef919 <unknown>
app       | #3 0x55ac01a1af16 <unknown>
app       | #4 0x55ac01a1717a <unknown>
app       | #5 0x55ac01a168a6 <unknown>
app       | #6 0x55ac0198f263 <unknown>
app       | #7 0x55ac01c503e4 <unknown>
app       | #8 0x55ac01c543d7 <unknown>
app       | #9 0x55ac01c5eb20 <unknown>
app       | #10 0x55ac01c55023 <unknown>
app       | #11 0x55ac01c231aa <unknown>
app       | #12 0x55ac0198da43 <unknown>
app       | #13 0x7f341189d18a <unknown>

某次崩溃还包含Chrome页面崩溃的相关信息。

我已尝试以下解决方案，但问题仍未解决：

• 调整/dev/shm挂载配置
• 修改Docker的shm-size参数
• 配置Chrome启动选项（无头模式、禁用沙箱等）
• 切换为Chromium浏览器

服务相关配置如下：

docker-compose配置片段 #

app:
# restart: always
build: ./app
links:
  - postgres:postgres
secrets:
  - postgres_user
  - postgres_password
volumes:
  - /dev/shm:/dev/shm
mem_limit: "2g"
mem_reservation: "512m"
shm_size: '2gb'
privileged: true
environment:
  POSTGRES_DB: db
  DATABASE_PORT: 5432
  POSTGRES_USER_FILE: /run/secrets/user
  POSTGRES_PASSWORD_FILE: /run/secrets/password
depends_on:
  - postgres
command: >
  sh -c "python3 app.py"

应用Dockerfile #

FROM python:3.10

COPY requirements.txt requirements.txt

RUN python -m pip install --upgrade pip && python -m pip install -r requirements.txt

RUN wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add -

RUN sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google-chrome.list'

RUN apt-get -y update
RUN apt-get install -y google-chrome-stable

RUN apt-get install -yqq unzip

RUN wget -O /tmp/chromedriver.zip http://chromedriver.storage.googleapis.com/`\
curl -sS chromedriver.storage.googleapis.com/LATEST_RELEASE\
`/chromedriver_linux64.zip

RUN unzip /tmp/chromedriver.zip chromedriver -d /usr/local/bin/

ENV DISPLAY=:99

COPY . .

ChromeOptions配置代码 #

def set_chrome_options() -> webdriver.ChromeOptions:
    """Sets chrome options for Selenium.
    Chrome options for headless browser is enabled.
    """
    chrome_options = webdriver.ChromeOptions()
    chrome_options.add_argument("--headless")
    chrome_options.add_argument("--no-sandbox")
    chrome_options.add_argument("--disable-dev-shm-usage")
    chrome_prefs = {}
    chrome_options.experimental_options["prefs"] = chrome_prefs
    chrome_prefs["profile.default_content_settings"] = {"images": 2}
    return chrome_options

确认爬取过程中未提前关闭driver，该服务在非Docker环境下运行正常，已在MacOS和Linux Mint 21系统测试，Docker版本为23.0.2。

内容的提问来源于stack exchange，提问作者Анатолий Смородин