使用buildx构建amd64镜像时Poetry导致体积暴增的问题

我在搭载M2 Pro的MacOS主机上构建Docker镜像，部署到EC2实例时遇到以下问题：

• 常规构建生成的镜像体积为2GB，但部署到EC2时出现兼容性警告：WARNING: The requested image's platform (linux/arm64/v8) does not match the detected host platform (linux/amd64/v3) and no specific platform was requested
• 使用buildx命令构建amd64架构镜像时，体积骤增至13GB，仅修改构建命令就出现巨大差异

我采用多阶段构建的Dockerfile，移除poetry config virtualenvs.create...命令后，amd64镜像体积能保持较小，推测Poetry是问题根源。调整pyproject.toml中torch依赖为amd64版本后，镜像体积降至10GB，但仍过大，通过du -h -d 1查看发现/app目录占用8.1GB。

Dockerfile #

FROM python:3.11-slim as builder

# Set environment variables to make Python and Poetry play nice
ENV POETRY_VERSION=1.7.1 \
    PYTHONUNBUFFERED=1 \
    PYTHONDONTWRITEBYTECODE=1 \

# for -slim version (it breaks if you don't comment out && apt-get clean)
RUN apt-get update && apt-get install -y \
    gfortran \
    libopenblas-dev \
    liblapack-dev \
    && apt-get clean \
    && rm -rf /var/lib/apt/lists/*

## Install poetry
RUN pip install "poetry==$POETRY_VERSION"

## copy project requirement files here to ensure they will be cached.
WORKDIR /app

COPY pyproject.toml ./

RUN poetry config virtualenvs.create false \
    && poetry install --no-interaction --no-dev --no-ansi  --verbose \
    && poetry cache clear pypi --all

FROM python:3.11-slim

# Copy all of the python files built in the Builder container into this smaller container.
COPY --from=builder /app /app
COPY --from=builder /usr/local/lib/python3.11 /usr/local/lib/python3.11

EXPOSE 7070

CMD ["poetry", "run", "flask", "run", "--host=0.0.0.0"]

常规构建命令（生成2GB镜像） #

docker build -f ./docker/Dockerfile \
-t malicious-url-prediction-img:v1 .

buildx构建命令（生成13GB镜像） #

docker buildx build --platform linux/amd64 -f ./docker/Dockerfile \
-t malicious-url-prediction-img:v1-amd64 .

修改后的pyproject.toml #

[tool.poetry]
name = "malicious-url"
version = "0.1.0"
description = ""
authors = ["Makoto1021 <makoto.miyazaki1021@gmail.com>"]
readme = "README.md"


[tool.poetry.dependencies]
python = "^3.11"
numpy = "^1.26.4"
tld = "^0.13"
fuzzywuzzy = "^0.18.0"
scikit-learn = "^1.4.1.post1"
pandas = "^2.2.1"
mlflow = {extras = ["pipelines"], version = "^2.11.3"}
xgboost = "^2.0.3"
python-dotenv = "^1.0.1"
imblearn = "^0.0"
flask = "^3.0.3"
torch = {url = "https://download.pytorch.org/whl/cpu-cxx11-abi/torch-2.2.2%2Bcpu.cxx11.abi-cp311-cp311-linux_x86_64.whl"}
googlesearch-python = "^1.2.3"
whois = "^1.20240129.2"
nltk = "^3.8.1"

[tool.poetry.group.dev.dependencies]
ipykernel = "^6.29.3"
tldextract = "^5.1.2"

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

磁盘占用情况 #

4.0K    ./mnt
1.9G    ./usr
4.0K    ./opt
4.0K    ./boot
0       ./sys
6.8M    ./var
4.0K    ./media
4.0K    ./tmp
1.4M    ./etc
4.0K    ./home
du: cannot access './proc/12/task/12/fd/7': No such file or directory
du: cannot access './proc/12/task/12/fdinfo/7': No such file or directory
du: cannot access './proc/12/fd/8': No such file or directory
du: cannot access './proc/12/fdinfo/8': No such file or directory
0       ./proc
8.0K    ./run
24K     ./root
0       ./dev
4.0K    ./srv
216K    ./utils
8.1G    ./app
10G     .

容器运行命令 #

docker run --rm -p 7070:5000 -v $(pwd)/logs:/app/logs malicious-url-prediction-img:v1-amd64

内容的提问来源于stack exchange，提问作者Makoto Miyazaki