Ubuntu24.04 Docker+Venv环境pip安装包路径优先级问题求助

基于ubuntu:24.04、Python 3.12.3和pip 25.1.1构建Docker镜像时，调用系统Python/pip安装包会触发以下错误：

error: externally-managed-environment 

改用venv管理Python包后，通过Singularity以非sudo用户运行容器（命令：singularity shell -H /home/davide/test docker://myImage，指定/home/davide/test为宿主目录），尝试安装额外包时，/opt/venv/bin/pip无法像之前那样回退到/home/davide/test/.local进行本地安装，反而报错：

pip3 install scanpy
ERROR: Will not install to the user site because it will lack sys.path precedence to numpy in /opt/venv/lib/python3.12/site-packages

可复现的最小Dockerfile #

FROM ubuntu:24.04

# general environmental variables
ENV DEBIAN_FRONTEND=noninteractive

SHELL ["/bin/bash", "-o", "pipefail", "-c"]
WORKDIR /

RUN apt-get update -y  && apt-get install -y python3 python3-venv python-is-python3 python3-pip

COPY config/requirements.txt /opt/requirements.txt
RUN python3 -m venv --system-site-packages /opt/venv && \
    /opt/venv/bin/pip install --no-cache-dir --upgrade pip setuptools && \
    /opt/venv/bin/pip install --no-cache-dir -r /opt/requirements.txt

ENV PATH=/opt/venv/bin:${PATH}
ENV PIP_USER=yes

ENTRYPOINT []

其中requirements.txt仅包含scipy。

容器启动后的环境信息 #

Singularity> python -m site
sys.path = [
    '/home/davide/test',
    '/usr/lib/python312.zip',
    '/usr/lib/python3.12',
    '/usr/lib/python3.12/lib-dynload',
    '/opt/venv/lib/python3.12/site-packages',
    '/usr/local/lib/python3.12/dist-packages',
    '/usr/lib/python3/dist-packages',
]
USER_BASE: '/home/davide/test/.local' (doesn't exist)
USER_SITE: '/home/davide/test/.local/lib/python3.12/site-packages' (doesn't exist)
ENABLE_USER_SITE: True

Singularity> which python
/opt/venv/bin/python


Singularity> ls -ltr /opt/venv/bin/
total 17
lrwxrwxrwx 1 root root    7 Jul 30 16:53 python3.12 -> python3
lrwxrwxrwx 1 root root   16 Jul 30 16:53 python3 -> /usr/bin/python3
lrwxrwxrwx 1 root root    7 Jul 30 16:53 python -> python3
-rw-r--r-- 1 root root 2180 Jul 30 16:53 activate.fish
-rw-r--r-- 1 root root  905 Jul 30 16:53 activate.csh
-rw-r--r-- 1 root root 2006 Jul 30 16:53 activate
-rw-r--r-- 1 root root 9033 Jul 30 16:53 Activate.ps1
-rwxr-xr-x 1 root root  226 Jul 30 16:53 pip3.12
-rwxr-xr-x 1 root root  226 Jul 30 16:53 pip3
-rwxr-xr-x 1 root root  226 Jul 30 16:53 pip
-rwxr-xr-x 1 root root  221 Jul 30 16:53 numpy-config
-rwxr-xr-x 1 root root  221 Jul 30 16:53 f2py

1. 问题根源 #

报错核心原因：

• 启用PIP_USER=yes后，pip默认安装到用户目录，但/opt/venv/lib/python3.12/site-packages在sys.path中的优先级高于用户站点目录
• 安装scanpy依赖的numpy已存在于venv中，若安装到用户目录，用户站点的包会因优先级低无法正常调用，pip为避免冲突触发报错

2. 方案一：临时调整环境变量并指定用户安装（推荐） #

在Singularity容器内执行以下命令安装：

unset PIP_USER && pip install --user scanpy

安装完成后，将用户站点目录添加到PYTHONPATH以提升优先级：

export PYTHONPATH="/home/davide/test/.local/lib/python3.12/site-packages:$PYTHONPATH"

可将上述export命令添加到宿主目录的.bashrc或.profile中，实现容器启动时自动生效。

3. 方案二：修改Dockerfile创建用户专属venv #

若希望容器默认支持用户本地安装，可修改Dockerfile，构建时创建普通用户并生成个人venv：

FROM ubuntu:24.04

ENV DEBIAN_FRONTEND=noninteractive
SHELL ["/bin/bash", "-o", "pipefail", "-c"]
WORKDIR /

RUN apt-get update -y  && apt-get install -y python3 python3-venv python-is-python3 python3-pip

# 创建普通用户
RUN useradd -m davide
USER davide

# 创建用户个人venv
RUN python3 -m venv ~/venv && \
    ~/venv/bin/pip install --no-cache-dir --upgrade pip setuptools && \
    ~/venv/bin/pip install --no-cache-dir scipy

ENV PATH="/home/davide/venv/bin:${PATH}"

ENTRYPOINT []

此方式下，Singularity启动容器后，用户可直接在个人venv中安装包，无需担心权限与优先级问题。

4. 方案三：强制忽略优先级安装（不推荐） #

若必须安装到用户目录且可接受潜在依赖冲突，可使用以下命令强制安装：

pip install --user --ignore-installed scanpy

内容的提问来源于stack exchange，提问作者cast