基于nvidia/cuda镜像安装Python3.11时遭遇html5lib导入错误求助

我基于nvidia/cuda:11.3.1-cudnn8-runtime-ubuntu20.04镜像编写了如下Dockerfile，通过deadsnakes PPA安装Python3.11：

FROM nvidia/cuda:11.3.1-cudnn8-runtime-ubuntu20.04
ENV DEBIAN_FRONTEND=noninteractive

# 修复镜像中过时的密钥
RUN apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/3bf863cc.pub

# 更新包列表并安装基础工具
RUN apt-get update && \
    apt-get install -y software-properties-common

# 添加deadsnakes PPA源
RUN add-apt-repository ppa:deadsnakes/ppa

RUN apt update --fix-missing && \
        apt install python3.11 python3.11-distutils -y && \
        apt install python3-pip -y && \
        ln -sf /usr/bin/python3.11 /usr/bin/python && \
        ln -sf /usr/bin/pip3 /usr/bin/pip

# 安装最新版pip
RUN curl -sS https://bootstrap.pypa.io/get-pip.py | python3.11

# 安装编译依赖
RUN apt-get update -y && apt-get install -y --no-install-recommends build-essential gcc \
                                        libsndfile1

ARG UNAME=user
ARG UID=1000
ARG GID=1000
RUN groupadd -g $GID -o $UNAME
RUN useradd -m -u $UID -g $GID -o -s /bin/bash $UNAME
USER $UNAME

COPY requirements.txt /tmp/requirements.txt
RUN python -m pip install --user setuptools wheel
RUN python -m pip install --user --no-cache-dir -r /tmp/requirements.txt -f https://download.pytorch.org/whl/cu113/torch_stable.html

执行镜像构建时，持续出现如下错误：

File "/usr/lib/python3/dist-packages/pip/_internal/index/collector.py", line 12, in <module>
0.929     from pip._vendor import html5lib, requests
0.929 ImportError: cannot import name 'html5lib' from 'pip._vendor' (/usr/lib/python3/dist-packages/pip/_vendor/__init__.py)

我尝试过用get-pip.py安装pip，但问题仍未解决。

修复方案 #

问题根源是系统自带的python3-pip与Python3.11版本不兼容，且后续安装的新版pip没有覆盖旧的系统pip。调整Dockerfile步骤即可解决：

修改后的Dockerfile #

FROM nvidia/cuda:11.3.1-cudnn8-runtime-ubuntu20.04
ENV DEBIAN_FRONTEND=noninteractive

# 修复镜像中过时的密钥
RUN apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/3bf863cc.pub

# 更新包列表并安装基础工具
RUN apt-get update && \
    apt-get install -y software-properties-common

# 添加deadsnakes PPA源
RUN add-apt-repository ppa:deadsnakes/ppa

# 安装Python3.11及distutils，跳过系统pip安装
RUN apt update --fix-missing && \
        apt install python3.11 python3.11-distutils -y && \
        ln -sf /usr/bin/python3.11 /usr/bin/python

# 给Python3.11安装专属pip，并设置正确软链接
RUN curl -sS https://bootstrap.pypa.io/get-pip.py | python3.11 && \
        ln -sf /usr/local/bin/pip3.11 /usr/bin/pip && \
        ln -sf /usr/local/bin/pip3.11 /usr/bin/pip3

# 安装编译依赖
RUN apt-get update -y && apt-get install -y --no-install-recommends build-essential gcc \
                                        libsndfile1

ARG UNAME=user
ARG UID=1000
ARG GID=1000
RUN groupadd -g $GID -o $UNAME
RUN useradd -m -u $UID -g $GID -o -s /bin/bash $UNAME
USER $UNAME

COPY requirements.txt /tmp/requirements.txt
RUN python -m pip install --user setuptools wheel
RUN python -m pip install --user --no-cache-dir -r /tmp/requirements.txt -f https://download.pytorch.org/whl/cu113/torch_stable.html

关键修改点 #

• 移除apt install python3-pip -y步骤，避免引入与Python3.11不兼容的系统pip
• 调整软链接，让pip和pip3指向Python3.11专属的/usr/local/bin/pip3.11，确保版本完全匹配
• 合并pip安装与软链接步骤，避免中间状态的版本冲突

内容的提问来源于stack exchange，提问作者Carlo