Ubuntu更新后Docker Python3.x-slim镜像构建遇python3.x-minimal权限错误

问题背景 #

应用原本运行正常，计划从Ubuntu 20迁移到新EC2实例的Ubuntu 22时，Docker构建失败；且Ubuntu 20自动更新后也出现同样问题。尝试将Python slim版本升级至3.10.x、3.11.x，问题仍未解决。

原Dockerfile内容 #

# pull official base image
FROM python:3.9.13-slim as builder

# set work directory
WORKDIR /usr/src/app

# set environment variables
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1

# install psycopg2 dependencies
RUN apt update \
    && apt install -y build-essential gcc python3-dev musl-dev libffi-dev libssl-dev postgresql-server-dev-all cargo git tk libmagic1 

# lint
RUN pip install filemagic
RUN pip install --upgrade pip
RUN pip install flake8
COPY . /usr/src/app/

# install dependencies
COPY ./requirements.txt .
RUN pip install wheel
RUN pip wheel --no-cache-dir --no-deps --wheel-dir /usr/src/app/wheels -r requirements.txt

#########
# FINAL #
#########

# pull official base image
FROM python:3.9.13-slim

# create directory for the app user
RUN mkdir -p /home/app

# create the app user
# RUN addgroup -S app && adduser -S app -G app
RUN adduser --system --group app

# create the appropriate directories
ENV HOME=/home/app
ENV APP_HOME=/home/app/web
RUN mkdir $APP_HOME
RUN mkdir $APP_HOME/staticfiles
WORKDIR $APP_HOME

# install dependencies
RUN apt update && apt install -y tk
COPY --from=builder /usr/src/app/wheels /wheels
COPY --from=builder /usr/src/app/requirements.txt .
RUN pip install --upgrade pip
RUN pip install --no-cache /wheels/*

# copy entrypoint.sh
COPY ./entrypoint.sh $APP_HOME

# copy project
COPY . $APP_HOME

# chown all the files to the app user
RUN chown -R app:app $APP_HOME

RUN chmod +x $APP_HOME/entrypoint.sh

# change to the app user
USER app

# run entrypoint.prod.sh
ENTRYPOINT ["/home/app/web/entrypoint.sh"]

构建错误日志 #

Preparing to unpack .../libpython3.9-minimal_3.9.2-1_amd64.deb ...
#0 26.70 Unpacking libpython3.9-minimal:amd64 (3.9.2-1) ...
#0 27.21 Selecting previously unselected package python3.9-minimal.
#0 27.21 Preparing to unpack .../python3.9-minimal_3.9.2-1_amd64.deb ...
#0 27.23 Unpacking python3.9-minimal (3.9.2-1) ...
#0 27.79 Setting up libpython3.9-minimal:amd64 (3.9.2-1) ...
#0 27.82 Setting up python3.9-minimal (3.9.2-1) ...
#0 28.16 Traceback (most recent call last):
#0 28.17   File "/usr/lib/python3.9/py_compile.py", line 215, in <module>
#0 28.17     sys.exit(main())
#0 28.17   File "/usr/lib/python3.9/py_compile.py", line 207, in main
#0 28.17     compile(filename, doraise=True)
#0 28.17   File "/usr/lib/python3.9/py_compile.py", line 172, in compile
#0 28.17     importlib._bootstrap_external._write_atomic(cfile, bytecode, mode)
#0 28.17   File "<frozen importlib._bootstrap_external>", line 126, in _write_atomic
#0 28.18 PermissionError: [Errno 13] Permission denied: '/usr/lib/python3.9/__pycache__/__future__.cpython-39.pyc.140199513461120'
#0 28.18 dpkg: error processing package python3.9-minimal (--configure):
#0 28.18  installed python3.9-minimal package post-installation script subprocess returned error exit status 1
#0 28.21 Errors were encountered while processing:
#0 28.21  python3.9-minimal
#0 28.37 E: Sub-process /usr/bin/dpkg returned an error code (1)
------
failed to solve: executor failed running [/bin/sh -c apt update     && apt install -y build-essential gcc python3-dev musl-dev libffi-dev libssl-dev postgresql-server-dev-all cargo git tk libmagic1]: exit code: 100
service "web" is not running container #1

解决方案 #

问题根源 #

builder阶段开头设置的ENV PYTHONDONTWRITEBYTECODE 1会阻止Python生成.pyc字节码文件。而apt install安装python3.9-minimal时，其post-install脚本会尝试编译系统Python库文件生成缓存，此时该环境变量会导致Python无法写入/usr/lib/python3.9/__pycache__目录，触发权限错误。

修复方案1：临时取消环境变量 #

在builder阶段的apt install命令前临时取消PYTHONDONTWRITEBYTECODE，安装完成后再恢复：

# pull official base image
FROM python:3.9.13-slim as builder

# set work directory
WORKDIR /usr/src/app

# 先只设置不影响系统包安装的环境变量
ENV PYTHONUNBUFFERED 1

# 临时取消PYTHONDONTWRITEBYTECODE，执行apt安装
RUN PYTHONDONTWRITEBYTECODE=0 apt update \
    && apt install -y build-essential gcc python3-dev musl-dev libffi-dev libssl-dev postgresql-server-dev-all cargo git tk libmagic1 

# 恢复环境变量
ENV PYTHONDONTWRITEBYTECODE 1

# lint
RUN pip install filemagic
RUN pip install --upgrade pip
RUN pip install flake8
COPY . /usr/src/app/

# install dependencies
COPY ./requirements.txt .
RUN pip install wheel
RUN pip wheel --no-cache-dir --no-deps --wheel-dir /usr/src/app/wheels -r requirements.txt

修复方案2：延迟设置环境变量 #

将ENV PYTHONDONTWRITEBYTECODE 1的设置时机调整到apt install完成之后，避免干扰系统包安装：

# pull official base image
FROM python:3.9.13-slim as builder

# set work directory
WORKDIR /usr/src/app

# 先只设置PYTHONUNBUFFERED
ENV PYTHONUNBUFFERED 1

# 执行apt安装
RUN apt update \
    && apt install -y build-essential gcc python3-dev musl-dev libffi-dev libssl-dev postgresql-server-dev-all cargo git tk libmagic1 

# 安装完成后再设置PYTHONDONTWRITEBYTECODE
ENV PYTHONDONTWRITEBYTECODE 1

# lint
RUN pip install filemagic
RUN pip install --upgrade pip
RUN pip install flake8
COPY . /usr/src/app/

# install dependencies
COPY ./requirements.txt .
RUN pip install wheel
RUN pip wheel --no-cache-dir --no-deps --wheel-dir /usr/src/app/wheels -r requirements.txt

关于升级Python版本无效的说明 #

无论使用Python 3.9/3.10/3.11，只要在apt install系统Python包前设置了PYTHONDONTWRITEBYTECODE=1，都会触发该问题——因为系统包的post-install脚本会调用系统Python编译缓存，该环境变量会干扰这个过程，与Python具体版本无关。

内容的提问来源于stack exchange，提问作者Baltschun Ali