TeamCity CI构建Python项目时抛出线程启动异常求助

问题场景 #

我通过TeamCity为部署在EKS中的Python项目执行CI流程，此前运行正常，但近期执行Dockerfile时，在pip install安装Python包步骤中抛出如下异常：

[12:23:06]  Traceback (most recent call last):
[12:23:06]    File "/usr/local/lib/python3.11/site-packages/pip/_internal/cli/base_command.py", line 169, in exc_logging_wrapper
[12:23:06]      status = run_func(*args)
[12:23:06]               ^^^^^^^^^^^^^^^
[12:23:06]    File "/usr/local/lib/python3.11/site-packages/pip/_internal/cli/req_command.py", line 248, in wrapper
[12:23:06]      return func(self, options, args)
[12:23:06]             ^^^^^^^^^^^^^^^^^^^^^^^^^
[12:23:06]    File "/usr/local/lib/python3.11/site-packages/pip/_internal/commands/install.py", line 377, in run
[12:23:06]      requirement_set = resolver.resolve(
[12:23:06]                        ^^^^^^^^^^^^^^^^^
[12:23:06]    File "/usr/local/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 92, in resolve
[12:23:06]      result = self._result = resolver.resolve(
[12:23:06]                              ^^^^^^^^^^^^^^^^^
[12:23:06]    File "/usr/local/lib/python3.11/site-packages/pip/_vendor/resolvelib/resolvers.py", line 546, in resolve
[12:23:06]      state = resolution.resolve(requirements, max_rounds=max_rounds)
[12:23:06]              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[12:23:06]    File "/usr/local/lib/python3.11/site-packages/pip/_vendor/resolvelib/resolvers.py", line 397, in resolve
[12:23:06]      self._add_to_criteria(self.state.criteria, r, parent=None)
[12:23:06]    File "/usr/local/lib/python3.11/site-packages/pip/_vendor/resolvelib/resolvers.py", line 173, in _add_to_criteria
[12:23:06]      if not criterion.candidates:
[12:23:06]    File "/usr/local/lib/python3.11/site-packages/pip/_vendor/resolvelib/structs.py", line 156, in __bool__
[12:23:06]      return bool(self._sequence)
[12:23:06]             ^^^^^^^^^^^^^^^^^^^^
[12:23:06]    File "/usr/local/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 155, in __bool__
[12:23:06]      return any(self)
[12:23:06]             ^^^^^^^^^
[12:23:06]    File "/usr/local/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 143, in <genexpr>
[12:23:06]      return (c for c in iterator if id(c) not in self._incompatible_ids)
[12:23:06]             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[12:23:06]    File "/usr/local/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 47, in _iter_built
[12:23:06]      candidate = func()
[12:23:06]                  ^^^^^^
[12:23:06]    File "/usr/local/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 206, in _make_candidate_from_link
[12:23:06]      self._link_candidate_cache[link] = LinkCandidate(
[12:23:06]                                         ^^^^^^^^^^^^^^
[12:23:06]    File "/usr/local/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 293, in __init__
[12:23:06]      super().__init__(
[12:23:06]    File "/usr/local/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 156, in __init__
[12:23:06]      self.dist = self._prepare()
[12:23:06]                  ^^^^^^^^^^^^^^^
[12:23:06]    File "/usr/local/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 225, in _prepare
[12:23:06]      dist = self._prepare_distribution()
[12:23:06]             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[12:23:06]    File "/usr/local/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 304, in _prepare_distribution
[12:23:06]      return preparer.prepare_linked_requirement(self._ireq, parallel_builds=True)
[12:23:06]             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[12:23:06]    File "/usr/local/lib/python3.11/site-packages/pip/_internal/operations/prepare.py", line 516, in prepare_linked_requirement
[12:23:06]      return self._prepare_linked_requirement(req, parallel_builds)
[12:23:06]             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[12:23:06]    File "/usr/local/lib/python3.11/site-packages/pip/_internal/operations/prepare.py", line 587, in _prepare_linked_requirement
[12:23:06]      local_file = unpack_url(
[12:23:06]                   ^^^^^^^^^^^
[12:23:06]    File "/usr/local/lib/python3.11/site-packages/pip/_internal/operations/prepare.py", line 166, in unpack_url
[12:23:06]      file = get_http_url(
[12:23:06]             ^^^^^^^^^^^^^
[12:23:06]    File "/usr/local/lib/python3.11/site-packages/pip/_internal/operations/prepare.py", line 107, in get_http_url
[12:23:06]      from_path, content_type = download(link, temp_dir.path)
[12:23:06]                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[12:23:06]    File "/usr/local/lib/python3.11/site-packages/pip/_internal/network/download.py", line 147, in __call__
[12:23:06]      for chunk in chunks:
[12:23:06]    File "/usr/local/lib/python3.11/site-packages/pip/_internal/cli/progress_bars.py", line 52, in _rich_progress_bar
[12:23:06]      with progress:
[12:23:06]    File "/usr/local/lib/python3.11/site-packages/pip/_vendor/rich/progress.py", line 1169, in __enter__
[12:23:06]      self.start()
[12:23:06]    File "/usr/local/lib/python3.11/site-packages/pip/_vendor/rich/progress.py", line 1160, in start
[12:23:06]      self.live.start(refresh=True)
[12:23:06]    File "/usr/local/lib/python3.11/site-packages/pip/_vendor/rich/live.py", line 132, in start
[12:23:06]      self._refresh_thread.start()
[12:23:06]    File "/usr/local/lib/python3.11/threading.py", line 957, in start
[12:23:06]      _start_new_thread(self._bootstrap, ())
[12:23:06]  RuntimeError: can't start new thread

错误分析 #

这个错误的核心是系统无法创建新线程，触发点是pip依赖的rich进度条组件尝试启动刷新线程时失败，常见诱因包括：

• Docker构建容器的线程数、内存配额过低，系统无法为新线程分配资源
• pip版本问题：部分pip版本与rich库的组合在资源受限环境下存在线程创建逻辑缺陷
• TeamCity构建代理的资源被调整，可用内存/线程数不足
• 依赖包过多，单次pip install占用资源过高

解决方案 #

1. 临时快速修复：禁用pip进度条 #

直接跳过进度条的线程创建逻辑，修改pip install命令：

pip install --no-progress-bar -r requirements.txt

这个方法能立即绕过问题，适合紧急恢复CI流程。

2. 调整Docker容器资源限制 #

在TeamCity的构建配置中，给Docker构建步骤增加内存和CPU分配，或者构建时显式指定资源参数：

docker build --memory=4g --cpus=2 -t your-project-image .

3. 调整pip版本 #

部分pip版本（如22.x系列）在资源受限环境下的线程处理有问题，可尝试降级到稳定版本：

pip install --upgrade pip==21.3.1

或者升级到最新版，验证是否修复了该问题：

pip install --upgrade pip

4. 优化依赖安装 #

• 检查requirements.txt，移除不必要的依赖，减少资源占用
• 使用--no-cache-dir参数避免缓存占用额外内存：

  pip install --no-cache-dir -r requirements.txt
  

• 拆分依赖安装，将大的依赖包单独安装，降低单次安装的资源负载

5. 检查TeamCity代理资源 #

确认TeamCity构建代理所在机器的空闲内存、线程数是否充足，是否有其他进程占用了大量资源。

内容的提问来源于stack exchange，提问作者Developer