基于NVIDIA PyTorch镜像构建Docker时pip安装依赖异常缓慢

以nvcr.io/nvidia/pytorch:23.05-py3作为基础镜像构建Docker镜像时，其他步骤均正常，但执行pip install --no-cache-dir -r requirements.txt安装Python依赖的步骤耗时超3小时仍未完成，构建过程中pip出现大量版本回溯与重试行为。

Dockerfile片段 #

FROM nvcr.io/nvidia/pytorch:23.05-py3

ENV LANG=C.UTF-8
ENV PYTHONUNBUFFERED=TRUE
ENV PYTHONDONTWRITEBYTECODE=TRUE
ENV NVIDIA_VISIBLE_DEVICES="all"

#install ffmpeg and sudo
RUN apt update && \
        apt install -y software-properties-common && \
        add-apt-repository -y ppa:jonathonf/ffmpeg-4 && \
        apt install -y ffmpeg && \
        apt install -y sudo && \
        rm -rf /var/lib/apt/lists/*

#install git-lfs
RUN curl -s https://packagecloud.io/install/repositories/github/git-lfs/script.deb.sh | bash && \
    apt-get install -y git-lfs && \
    git lfs install

#install requirements
COPY requirements.txt requirements.txt
RUN pip install --no-cache-dir -r requirements.txt && \
        pip uninstall -y jupyterlab_widgets && \
        pip install jupyterlab_widgets

requirements.txt内容 #

boto3==1.38.32
librosa==0.11.0
evaluate>=0.4.0
jiwer>=3.10
gradio>=5.3.0
bitsandbytes==0.46.0
datasets==3.5.0
accelerate==1.6.0
loralib==0.1.2
transformers==4.51.3
peft==0.15.2
sagemaker-pytorch-training>=2.8.0

构建日志片段 #

=> [5/5] RUN pip install --no-cache-dir -r requirements.txt &&         pip uninstall -y jupyterlab_widgets &&         pip install jupyterlab_widgets                           25323.0s 
 => => #   Downloading charset_normalizer-2.1.0-py3-none-any.whl (39 kB)                                                                                                                
 => => # INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. If you want to abort this run, you can p
 => => # ress Ctrl + C to do so. To improve how pip performs, tell us what happened here: https://pip.pypa.io/surveys/backtracking                                                      
 => => #   Downloading charset_normalizer-2.0.12-py3-none-any.whl (39 kB)                                                                                                               
 => => #   Downloading charset_normalizer-2.0.11-py3-none-any.whl (39 kB)                                                                                                               
 => => #   Downloading charset_normalizer-2.0.10-py3-none-any.whl (39 kB)

解决方案 #

1. 升级pip版本
   基础镜像中的pip版本可能较旧，依赖解析器效率低下。在安装requirements前先升级pip：

   RUN pip install --upgrade pip && \
       pip install --no-cache-dir -r requirements.txt && \
       pip uninstall -y jupyterlab_widgets && \
       pip install jupyterlab_widgets
   

2. 切换国内pip镜像源
   替换默认源为国内镜像加速下载，同时避免网络波动导致的重试：

   RUN pip install --upgrade pip -i https://pypi.tuna.tsinghua.edu.cn/simple && \
       pip install --no-cache-dir -r requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple && \
       pip uninstall -y jupyterlab_widgets && \
       pip install jupyterlab_widgets -i https://pypi.tuna.tsinghua.edu.cn/simple
   

3. 明确所有依赖的具体版本
   requirements.txt中多个包使用>=模糊约束，会导致pip反复回溯寻找兼容版本。将这些包改为具体版本：

   boto3==1.38.32
   librosa==0.11.0
   evaluate==0.4.0
   jiwer==3.10
   gradio==5.3.0
   bitsandbytes==0.46.0
   datasets==3.5.0
   accelerate==1.6.0
   loralib==0.1.2
   transformers==4.51.3
   peft==0.15.2
   sagemaker-pytorch-training==2.8.0
   

4. 优化jupyterlab_widgets的安装步骤
   无需先卸载再重装，直接指定版本安装即可，减少冗余操作：

   RUN pip install --upgrade pip -i https://pypi.tuna.tsinghua.edu.cn/simple && \
       pip install --no-cache-dir -r requirements.txt jupyterlab_widgets -i https://pypi.tuna.tsinghua.edu.cn/simple
   

5. 拆分依赖安装步骤
   将冲突风险高的包分开安装，降低pip解析复杂度：

   RUN pip install --upgrade pip -i https://pypi.tuna.tsinghua.edu.cn/simple
   # 先安装核心ML依赖
   RUN pip install --no-cache-dir transformers==4.51.3 peft==0.15.2 accelerate==1.6.0 bitsandbytes==0.46.0 -i https://pypi.tuna.tsinghua.edu.cn/simple
   # 再安装其他依赖
   RUN pip install --no-cache-dir -r requirements.txt jupyterlab_widgets -i https://pypi.tuna.tsinghua.edu.cn/simple
   

内容的提问来源于stack exchange，提问作者THANAWUT TIMPITAK