Docker构建Python镜像时pip安装触发RuntimeError: can't start new thread

问题重现 #

使用以下Dockerfile构建Python镜像：

# Use an official Python 3.9 image as the base image
FROM python:3.9

# Set the working directory to /app
WORKDIR /app

# Copy the requirements.txt file into the container at /app
COPY requirements.txt /app/

# Install any needed packages specified in requirements.txt
RUN pip install -r requirements.txt

# Copy the web.py file into the container at /app
COPY web.py /app/

# Make port 7860 available to the world outside this container
EXPOSE 7860

# Define the command to run your application
CMD ["python", "web.py"]

执行构建命令：

docker build -t myimage .

出现如下错误：

=> ERROR [webapp 4/5] RUN pip install -r requirements.txt                                                                                                                     2.2s
------
 > [webapp 4/5] RUN pip install -r requirements.txt:
1.804 Collecting huggingface_hub==0.18.0
1.876   Downloading huggingface_hub-0.18.0-py3-none-any.whl (301 kB)
1.886 ERROR: Exception:
1.886 Traceback (most recent call last):
1.886   File "/usr/local/lib/python3.9/site-packages/pip/_internal/cli/base_command.py", line 160, in exc_logging_wrapper
1.886     status = run_func(*args)
1.886   File "/usr/local/lib/python3.9/site-packages/pip/_internal/cli/req_command.py", line 247, in wrapper
1.886     return func(self, options, args)
1.886   File "/usr/local/lib/python3.9/site-packages/pip/_internal/commands/install.py", line 419, in run
1.886     requirement_set = resolver.resolve(
1.886   File "/usr/local/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 92, in resolve
1.886     result = self._result = resolver.resolve(
1.886   File "/usr/local/lib/python3.9/site-packages/pip/_vendor/resolvelib/resolvers.py", line 481, in resolve
1.886     state = resolution.resolve(requirements, max_rounds=max_rounds)
1.886   File "/usr/local/lib/python3.9/site-packages/pip/_vendor/resolvelib/resolvers.py", line 348, in resolve
1.886     self._add_to_criteria(self.state.criteria, r, parent=None)
1.886   File "/usr/local/lib/python3.9/site-packages/pip/_vendor/resolvelib/resolvers.py", line 172, in _add_to_criteria
1.886     if not criterion.candidates:
1.886   File "/usr/local/lib/python3.9/site-packages/pip/_vendor/resolvelib/structs.py", line 151, in __bool__
1.886     return bool(self._sequence)
1.886   File "/usr/local/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 155, in __bool__
1.886     return any(self)
1.886   File "/usr/local/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 143, in <genexpr>
1.886     return (c for c in iterator if id(c) not in self._incompatible_ids)
1.886   File "/usr/local/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 47, in _iter_built
1.886     candidate = func()
1.886   File "/usr/local/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 206, in _make_candidate_from_link
1.886     self._link_candidate_cache[link] = LinkCandidate(
1.886   File "/usr/local/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 297, in __init__
1.886     super().__init__(
1.886   File "/usr/local/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 162, in __init__
1.886     self.dist = self._prepare()
1.886   File "/usr/local/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 231, in _prepare
1.886     dist = self._prepare_distribution()
1.886   File "/usr/local/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 308, in _prepare_distribution
1.886     return preparer.prepare_linked_requirement(self._ireq, parallel_builds=True)
1.886   File "/usr/local/lib/python3.9/site-packages/pip/_internal/operations/prepare.py", line 491, in prepare_linked_requirement
1.886     return self._prepare_linked_requirement(req, parallel_builds)
1.886   File "/usr/local/lib/python3.9/site-packages/pip/_internal/operations/prepare.py", line 536, in _prepare_linked_requirement
1.886     local_file = unpack_url(
1.886   File "/usr/local/lib/python3.9/site-packages/pip/_internal/operations/prepare.py", line 166, in unpack_url
1.886     file = get_http_url(
1.886   File "/usr/local/lib/python3.9/site-packages/pip/_internal/operations/prepare.py", line 107, in get_http_url
1.886     from_path, content_type = download(link, temp_dir.path)
1.886   File "/usr/local/lib/python3.9/site-packages/pip/_internal/network/download.py", line 147, in __call__
1.886     for chunk in chunks:
1.886   File "/usr/local/lib/python3.9/site-packages/pip/_internal/cli/progress_bars.py", line 52, in _rich_progress_bar
1.886     with progress:
1.886   File "/usr/local/lib/python3.9/site-packages/pip/_vendor/rich/progress.py", line 1169, in __enter__
1.886     self.start()
1.886   File "/usr/local/lib/python3.9/site-packages/pip/_vendor/rich/progress.py", line 1160, in start
1.886     self.live.start(refresh=True)
1.886   File "/usr/local/lib/python3.9/site-packages/pip/_vendor/rich/live.py", line 132, in start
1.886     self._refresh_thread.start()
1.886   File "/usr/local/lib/python3.9/threading.py", line 899, in start
1.886     _start_new_thread(self._bootstrap, ())
1.886 RuntimeError: can't start new thread
2.040
2.040 [notice] A new release of pip is available: 23.0.1 -> 23.3.1
2.040 [notice] To update, run: pip install --upgrade pip
------
failed to solve: executor failed running [/bin/sh -c pip install -r requirements.txt]: exit code: 2

环境信息 #

• 构建系统：Ubuntu 18.04
• Docker版本：Docker version 20.10.7, build f0df350
• Docker Compose版本：v2.23.0

解决方案 #

1. 禁用pip的进度条 #

pip下载包时启动的进度条线程触发了问题，直接禁用即可：
修改Dockerfile中的pip安装命令：

RUN pip install --no-progress-bar -r requirements.txt

或通过环境变量全局关闭：

ENV PIP_PROGRESS_BAR off
RUN pip install -r requirements.txt

2. 升级pip到指定版本 #

旧版本pip的rich进度条组件存在线程问题，先升级pip再安装依赖：

RUN pip install --upgrade pip==23.3.1 && pip install -r requirements.txt

3. 切换Python基础镜像 #

官方python:3.9镜像可能存在资源限制或依赖冗余，换用轻量版镜像：

FROM python:3.9-slim

slim版本体积更小、默认依赖更少，可降低线程冲突概率。

4. 调整容器构建时的线程资源限制 #

通过--ulimit参数设置容器最大进程/线程数，避免系统资源限制：

docker build --ulimit nproc=1024:2048 -t myimage .

5. 使用国内PyPI镜像源加速下载 #

若网络延迟间接导致线程超时，切换国内镜像源提升下载效率：

RUN pip install -i https://pypi.tuna.tsinghua.edu.cn/simple -r requirements.txt

内容的提问来源于stack exchange，提问作者Daniil Yefimov