Jenkins Docker环境下pip升级报错：无法启动新线程求助

问题背景 #

在Jenkins流水线构建Docker镜像时，Dockerfile中执行pip install --upgrade pip步骤触发RuntimeError: can't start new thread错误，导致构建失败。

错误日志 #

Step 1/9 : FROM python:3.9
 ---> 60d390c1959b
Step 2/9 : COPY AWS_Lambda /app
 ---> Using cache
 ---> f503fa1503cc
Step 3/9 : WORKDIR /app
 ---> Using cache
 ---> d6c217ca3c17
Step 4/9 : RUN pip uninstall -y pygoogle
 ---> Running in f25e6a69e9b9
WARNING: Skipping pygoogle as it is not installed.
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
Removing intermediate container f25e6a69e9b9
 ---> 436d544e2585
Step 5/9 : RUN pip install --upgrade pip
 ---> Running in 57790ab15a80
Requirement already satisfied: pip in /usr/local/lib/python3.9/site-packages (23.0.1)
Collecting pip
  Downloading pip-23.2.1-py3-none-any.whl (2.1 MB)
ERROR: Exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/pip/_internal/cli/base_command.py", line 160, in exc_logging_wrapper
    status = run_func(*args)
  File "/usr/local/lib/python3.9/site-packages/pip/_internal/cli/req_command.py", line 247, in wrapper
    return func(self, options, args)
  File "/usr/local/lib/python3.9/site-packages/pip/_internal/commands/install.py", line 419, in run
    requirement_set = resolver.resolve(
  File "/usr/local/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 92, in resolve
    result = self._result = resolver.resolve(
  File "/usr/local/lib/python3.9/site-packages/pip/_vendor/resolvelib/resolvers.py", line 481, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
  File "/usr/local/lib/python3.9/site-packages/pip/_vendor/resolvelib/resolvers.py", line 348, in resolve
    self._add_to_criteria(self.state.criteria, r, parent=None)
  File "/usr/local/lib/python3.9/site-packages/pip/_vendor/resolvelib/resolvers.py", line 172, in _add_to_criteria
    if not criterion.candidates:
  File "/usr/local/lib/python3.9/site-packages/pip/_vendor/resolvelib/structs.py", line 151, in __bool__
    return bool(self._sequence)
  File "/usr/local/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 155, in __bool__
    return any(self)
  File "/usr/local/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 143, in <genexpr>
    return (c for c in iterator if id(c) not in self._incompatible_ids)
  File "/usr/local/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 97, in _iter_built_with_inserted
    candidate = func()
  File "/usr/local/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 206, in _make_candidate_from_link
    self._link_candidate_cache[link] = LinkCandidate(
  File "/usr/local/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 297, in __init__
    super().__init__(
  File "/usr/local/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 162, in __init__
    self.dist = self._prepare()
  File "/usr/local/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 231, in _prepare
    dist = self._prepare_distribution()
  File "/usr/local/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 308, in _prepare_distribution
    return preparer.prepare_linked_requirement(self._ireq, parallel_builds=True)
  File "/usr/local/lib/python3.9/site-packages/pip/_internal/operations/prepare.py", line 491, in prepare_linked_requirement
    return self._prepare_linked_requirement(req, parallel_builds)
  File "/usr/local/lib/python3.9/site-packages/pip/_internal/operations/prepare.py", line 536, in _prepare_linked_requirement
    local_file = unpack_url(
  File "/usr/local/lib/python3.9/site-packages/pip/_internal/operations/prepare.py", line 166, in unpack_url
    file = get_http_url(
  File "/usr/local/lib/python3.9/site-packages/pip/_internal/operations/prepare.py", line 107, in get_http_url
    from_path, content_type = download(link, temp_dir.path)
  File "/usr/local/lib/python3.9/site-packages/pip/_internal/network/download.py", line 147, in __call__
    for chunk in chunks:
  File "/usr/local/lib/python3.9/site-packages/pip/_internal/cli/progress_bars.py", line 52, in _rich_progress_bar
    with progress:
  File "/usr/local/lib/python3.9/site-packages/pip/_vendor/rich/progress.py", line 1169, in __enter__
    self.start()
  File "/usr/local/lib/python3.9/site-packages/pip/_vendor/rich/progress.py", line 1160, in start
    self.live.start(refresh=True)
  File "/usr/local/lib/python3.9/site-packages/pip/_vendor/rich/live.py", line 132, in start
    self._refresh_thread.start()
  File "/usr/local/lib/python3.9/threading.py", line 899, in start
    _start_new_thread(self._bootstrap, ())
RuntimeError: can't start new thread

[notice] A new release of pip is available: 23.0.1 -> 23.2.1
[notice] To update, run: pip install --upgrade pip
The command '/bin/sh -c pip install --upgrade pip' returned a non-zero code: 2

解决方案 #

1. 禁用pip进度条 #

报错根源是pip依赖的rich进度条会启动新线程，在Jenkins构建环境的资源限制下无法创建线程。通过禁用进度条避免线程创建：
修改Dockerfile中的升级命令为：

RUN pip install --upgrade pip --no-progress-bar

或者通过环境变量全局禁用：

RUN PIP_PROGRESS_BAR=off pip install --upgrade pip

2. 使用预装高版本pip的基础镜像 #

直接选择已包含新版pip的Python镜像，省去构建时升级步骤。例如：

FROM python:3.9.18-slim

该版本镜像通常预装了较新版本的pip，无需额外升级。

3. 跳过pip升级 #

当前使用的pip 23.0.1已是稳定版本，若项目无强制升级要求，可直接删除Dockerfile中RUN pip install --upgrade pip步骤，减少构建环节的潜在问题。

4. 调整Jenkins构建环境的线程限制 #

若Jenkins所在容器线程资源不足，可修改容器启动参数放宽限制：

• 启动Jenkins容器时添加--ulimit nproc=65535:65535参数，提升进程/线程数量上限
• 联系管理员调整宿主机的线程限制配置

内容的提问来源于stack exchange，提问作者Abdul Shaikh