Azure Pipeline微软托管代理无法保留Docker中间镜像的问题及解决方法

问题描述 #

【2023年9月20日更新】问题出现在同一条流水线运行过程中，并非微软托管代理在不同运行间不保留数据导致。

【原始问题】我们有一个在自托管代理上运行正常的Azure DevOps YAML流水线，切换到微软托管代理后，无法获取之前构建阶段生成的单元测试Docker中间镜像。流水线中的脚本无法获取单元测试镜像，docker images --filter "label=unittestlayer=true" -q返回空结果，docker images -a的输出中也没有中间Docker镜像。

相关代码 #

Dockerfile

# Phase 1: Build the source code
FROM nginx:latest AS build
WORKDIR /src
RUN echo "Build completed" > build.txt

# Phase 2: Run unit tests
FROM build as testrunner
WORKDIR /testresults
RUN echo "Placeholder" > placeholder.txt
RUN echo "Unit test completed" > testresults.txt
LABEL unittestlayer=true

# Phase 3: Publish the binary
FROM build as publish
WORKDIR /publish
COPY --from=build /src/build.txt .
RUN echo "Binary publish completed" > publish.txt

# Phase 4: Build the final docker image
FROM nginx:latest AS base
WORKDIR /app
COPY --from=publish /publish .
COPY --from=testrunner /testresults/placeholder.txt .

azure-pipelines.yml

trigger:
  - main

pool:
  vmImage: "ubuntu-22.04"

steps:
  - task: DockerCompose@0
    displayName: "Build and Test"
    inputs:
      dockerComposeFile: "**/docker-compose.yml"
      dockerComposeFileArgs: |
        TAG=$(Build.BuildNumber)
      projectName: "demo"
      action: "Build services"

  - script: |
      docker images -a
      export unittestlayerid=$(docker images --filter "label=unittestlayer=true" -q)
      docker create --name unittestcontainer $unittestlayerid
      docker cp unittestcontainer:/testresults/ $(Build.ArtifactStagingDirectory)/testresults/
      docker stop unittestcontainer
      docker rm unittestcontainer
    displayName: "Retrieve Test and Coverage Results"
    continueOnError: false

原因分析 #

微软托管代理上的Docker Compose任务（DockerCompose@0）默认启用BuildKit构建模式，BuildKit为优化构建效率，默认会在构建完成后自动清理所有中间镜像。而自托管代理若未开启BuildKit，构建生成的中间镜像会被保留，因此后续脚本能正常获取。

解决方案 #

方法1：禁用BuildKit保留中间镜像 #

在DockerCompose@0任务中添加环境变量，关闭BuildKit的自动清理特性：

- task: DockerCompose@0
  displayName: "Build and Test"
  inputs:
    dockerComposeFile: "**/docker-compose.yml"
    dockerComposeFileArgs: |
      TAG=$(Build.BuildNumber)
    projectName: "demo"
    action: "Build services"
  env:
    DOCKER_BUILDKIT: 0

方法2：直接构建测试镜像并导出结果 #

跳过Docker Compose任务，使用原生docker build命令构建测试阶段镜像，直接运行容器并复制结果：

- script: |
    docker build --target testrunner -t testrunner-image .
    docker run --name test-container testrunner-image
    docker cp test-container:/testresults/ $(Build.ArtifactStagingDirectory)/testresults/
    docker rm test-container
  displayName: "Build Test Stage and Retrieve Results"

方法3：利用BuildKit特性直接导出结果（推荐） #

保留BuildKit的高效构建优势，通过BuildKit的绑定挂载和输出功能，在构建阶段直接将测试结果导出到宿主机：

1. 修改Dockerfile的testrunner阶段：

# Phase 2: Run unit tests
FROM build as testrunner
WORKDIR /testresults
RUN echo "Placeholder" > placeholder.txt
RUN echo "Unit test completed" > testresults.txt
LABEL unittestlayer=true
# 绑定宿主机目录并复制测试结果
RUN --mount=type=bind,target=/host-testresults \
    cp /testresults/* /host-testresults/

2. 更新流水线脚本：

- script: |
    mkdir -p $(Build.ArtifactStagingDirectory)/testresults
    DOCKER_BUILDKIT=1 docker build --target testrunner --output type=local,dest=$(Build.ArtifactStagingDirectory)/testresults .
  displayName: "Build with BuildKit and Export Test Results"

内容的提问来源于stack exchange，提问作者James L.