在AWS CodePipeline中使用Docker Buildx构建多平台镜像失败求助

I'm trying to build a multi-architecture Docker image (for linux/amd64, linux/arm64, linux/arm/v7, targeting Graviton2) using AWS CodePipeline. My original Build Spec works fine for single-arch builds, but when I replace docker build with docker buildx build --platform linux/amd64,linux/arm64,linux/arm/v7, the build fails with exit status 125.

Working Build Spec (single-arch) #

#########
# Build Spec #
# The build spec is used to build the image in code deploy. When using AWS
# CodePipeline, use this customized buildspec.
#
#########
version: 0.2
run-as: root
artifacts:
  files:
    - Dockerrun.aws.json
    - imagedefinitions.json
phases:
  install:
    runtime-versions:
      php: 7.4
  build:
    commands:
      - echo Build started on `date`
      - cp app/config/config.sample.php app/config/config.php
  post_build:
    commands:
      - echo Build completed on `date`
      - which aws
      - AWS_PASSWORD="$(aws ecr get-login-password --region us-east-1)"
      - docker build -t live -f docker/live/Dockerfile .
      - docker login -u AWS -p $AWS_PASSWORD xxxxxxxxxxx.dkr.ecr.us-east-1.amazonaws.com
      - docker tag live:latest xxxxxxxxxxx.dkr.ecr.us-east-1.amazonaws.com/live:latest
      - docker push xxxxxxxx.dkr.ecr.us-east-1.amazonaws.com/live:latest
      - mv docker/Dockerrun.aws.json Dockerrun.aws.json
      - echo Pushing the Docker image...
      - echo Writing image definitions file...
      - printf '[{"name":"live","imageUri":"%s"}]' xxxxxxxxxx.dkr.ecr.us-east-1.amazonaws.com/live:latest > imagedefinitions.json

Modified (Failing) Build Spec #

#########
# Build Spec #
# The build spec is used to build the image in code deploy. When using AWS
# CodePipeline, use this customized buildspec.
#
#########
version: 0.2
run-as: root
artifacts:
  files:
    - Dockerrun.aws.json
    - imagedefinitions.json
phases:
  install:
    runtime-versions:
      php: 7.4
  build:
    commands:
      - echo Build started on `date`
      - cp app/config/config.sample.php app/config/config.php
  post_build:
    commands:
      - echo Build completed on `date`
      - which aws
      - AWS_PASSWORD="$(aws ecr get-login-password --region us-east-1)"
      - docker buildx build --platform linux/amd64,linux/arm64,linux/arm/v7 -t live -f docker/live/Dockerfile .
      - docker login -u AWS -p $AWS_PASSWORD xxxxxxxxxxx.dkr.ecr.us-east-1.amazonaws.com
      - docker tag live:latest xxxxxxxxxxx.dkr.ecr.us-east-1.amazonaws.com/live:latest
      - docker push xxxxxxxx.dkr.ecr.us-east-1.amazonaws.com/live:latest
      - mv docker/Dockerrun.aws.json Dockerrun.aws.json
      - echo Pushing the Docker image...
      - echo Writing image definitions file...
      - printf '[{"name":"live","imageUri":"%s"}]' xxxxxxxxxx.dkr.ecr.us-east-1.amazonaws.com/live:latest > imagedefinitions.json

Error Message #

[Container] 2020/11/09 00:19:02 Phase context status code: COMMAND_EXECUTION_ERROR Message: Error while executing command: docker buildx build --platform linux/amd64,linux/arm64,linux/arm/v7 -t live -f docker/live/Dockerfile . Reason: exit status 125

What's wrong with my setup that's causing Docker Buildx to fail with exit status 125?

Exit status 125 in Docker usually points to issues with the command execution environment or invalid arguments. Let's break down the problems in your setup and fix them step by step:

1. Docker Buildx isn't initialized in the CodeBuild environment #

AWS CodeBuild's default Docker environments don't come with Buildx pre-configured. You need to enable Docker's experimental features and create a dedicated builder instance for multi-arch builds.

Add these commands to your install phase before running buildx build:

# Enable Docker experimental features required for buildx
- echo '{"experimental": true}' > /root/.docker/config.json
# Create and set a multi-arch builder as active
- docker buildx create --use --name multiarch_builder
# Bootstrap the builder to set up QEMU emulators for cross-compilation
- docker buildx inspect multiarch_builder --bootstrap

2. Buildx doesn't load images to the local daemon by default #

Unlike docker build, docker buildx build doesn't automatically save built images to your local Docker daemon. That means your subsequent docker tag and docker push commands fail because the live:latest image doesn't exist locally.

The fix is to push directly to ECR from the buildx command using the --push flag, and skip the separate tag/push steps. Modify your buildx command to:

- docker buildx build --platform linux/amd64,linux/arm64,linux/arm/v7 -t xxxxxxxxxxx.dkr.ecr.us-east-1.amazonaws.com/live:latest --push -f docker/live/Dockerfile .

3. Enable privileged mode in CodeBuild #

Buildx needs privileged access to run QEMU emulators for cross-compiling to ARM architectures. Enable this in your CodeBuild project settings:

• Go to your CodeBuild project in the AWS Console
• Under Environment > Privileged, toggle the setting to Enabled

Updated Working Build Spec #

Here's the corrected build spec incorporating all these fixes:

#########
# Build Spec #
# The build spec is used to build the image in code deploy. When using AWS
# CodePipeline, use this customized buildspec.
#
#########
version: 0.2
run-as: root
artifacts:
  files:
    - Dockerrun.aws.json
    - imagedefinitions.json
phases:
  install:
    runtime-versions:
      php: 7.4
    commands:
      # Configure Docker for buildx
      - echo '{"experimental": true}' > /root/.docker/config.json
      - docker buildx create --use --name multiarch_builder
      - docker buildx inspect multiarch_builder --bootstrap
  build:
    commands:
      - echo Build started on `date`
      - cp app/config/config.sample.php app/config/config.php
  post_build:
    commands:
      - echo Build completed on `date`
      - which aws
      - AWS_PASSWORD="$(aws ecr get-login-password --region us-east-1)"
      - docker login -u AWS -p $AWS_PASSWORD xxxxxxxxxxx.dkr.ecr.us-east-1.amazonaws.com
      # Build and push multi-arch image directly via buildx
      - docker buildx build --platform linux/amd64,linux/arm64,linux/arm/v7 -t xxxxxxxxxxx.dkr.ecr.us-east-1.amazonaws.com/live:latest --push -f docker/live/Dockerfile .
      - mv docker/Dockerrun.aws.json Dockerrun.aws.json
      - echo Pushing the Docker image...
      - echo Writing image definitions file...
      - printf '[{"name":"live","imageUri":"%s"}]' xxxxxxxxxx.dkr.ecr.us-east-1.amazonaws.com/live:latest > imagedefinitions.json

Quick Notes #

• The --bootstrap flag ensures the builder sets up all necessary emulators for ARM builds.
• ECR natively supports multi-arch images, so pushing via buildx will create a manifest list that automatically serves the correct architecture image to clients.

内容的提问来源于stack exchange，提问作者Devin Dixon