Bitbucket Pipeline推送镜像至DO仓库后拉取旧镜像问题求助

问题背景 #

在Bitbucket Pipeline中实现Git Push触发的流程：构建Docker镜像 → 推送至Digital Ocean（DO）容器仓库 → 部署到DO Kubernetes环境。当前遇到的问题是：推送镜像后通过sleep(60)等待同步，但K8s仍频繁拉取旧镜像，需重新运行Pipeline才能获取新镜像。现有Pipeline配置见下方：

image: atlassian/default-image:3

options:
size: 1x

definitions:
services:
docker:
memory: 2048

pipelines:
#  custom:
#    dev-k8s:
#      - step:
#          name: Manual Build for Develop
#         services:
#           - docker
#          script
#            - docker build --build-arg VERSION=$VERSION_LABEL -t $IMAGE_TAG | tee -a logs.txt 2>&1
#          after-script:
#            - if [[ BITBUCKET_EXIT_CODE -eq 0 ]]; then exit 0; else echo "Step failed"; fi
#            - echo "Step failed"
#            - pipe: atlassian/slack-notify:2.1.0
#              variables:
#                WEBHOOK_URL: $SLACK_NOTIFICATION_URL_BACKEND
#                MESSAGE: '$(cat logs.txt)'
branches:
beta-k8s:
- step:
    name: Build for Beta & Push Image To Beta DigitalOcean
    services:
        - docker
    script:
        - set -o pipefail # important! makes sure the build fails when ANY of piped commands fail
        - export VERSION_LABEL=`[[ $BITBUCKET_TAG ]] && echo "$BITBUCKET_TAG-${BITBUCKET_COMMIT::7}" || echo "$BITBUCKET_REPO_FULL_NAME-${BITBUCKET_COMMIT::7}"`
        - export VERSION_REF=`[[ $BITBUCKET_TAG ]] && echo "$BITBUCKET_TAG" || echo "$BITBUCKET_BRANCH"`
        - export BUILD_NUMBER=`[[ $BITBUCKET_BUILD_NUMBER ]] && echo "$BITBUCKET_BUILD_NUMBER"`
        - cd deployment/kubernetes && docker build --build-arg VERSION=$VERSION_LABEL -t xxxxx-backend:beta-build-$BUILD_NUMBER -t xxxxx-backend:beta -f ./Dockerfile ../.. | tee -a logs.txt 2>&1
        - export REMOTE_IMAGE_TAG="registry.digitalocean.com/xxxxx/k8s-xxxxx-backend:beta" && echo REMOTE_IMAGE_TAG
        - export REMOTE_IMAGE_TAG_W_BUILD="registry.digitalocean.com/xxxxx/k8s-xxxxx-backend:beta-build-$BUILD_NUMBER" && echo REMOTE_IMAGE_TAG_W_BUILD
        - docker login -u $DIGITALOCEAN_CONTAINER_REGISTRY_TOKEN -p $DIGITALOCEAN_CONTAINER_REGISTRY_TOKEN registry.digitalocean.com | tee -a logs.txt 2>&1
        - docker tag xxxxx-backend:beta $REMOTE_IMAGE_TAG | tee -a logs.txt 2>&1
        - docker tag xxxxx-backend:beta $REMOTE_IMAGE_TAG_W_BUILD | tee -a logs.txt 2>&1
        - docker push $REMOTE_IMAGE_TAG | tee -a logs.txt 2>&1
        - docker push $REMOTE_IMAGE_TAG_W_BUILD | tee -a logs.txt 2>&1
        - echo "Waiting for a minute..."; sleep 60
- step:
    name: Refresh Beta Kubernetes
    trigger: automatic  # This is the problem. Should run on hook I think
    script:
        - pipe: atlassian/kubectl-run:3.6.0
          variables:
                KUBE_CONFIG: $KUBE_CONFIG
                KUBECTL_COMMAND: 'rollout restart deployment backend-beta backend-queue-default-beta backend-queue-notifications-beta'

核心问题分析 #

1. 浮标标签导致的缓存问题：使用beta这类固定标签（浮标标签）时，K8s默认镜像拉取策略为IfNotPresent——节点本地已有该标签的镜像时，不会主动拉取仓库中更新的同标签镜像。
2. 固定sleep不可靠：DO容器仓库的镜像同步延迟不固定，60秒等待无法保证镜像完全可用，仍可能出现拉取旧镜像的情况。

解决方案 #

1. 使用唯一标签部署（推荐） #

利用已构建的beta-build-$BUILD_NUMBER唯一标签部署，彻底规避缓存问题：

• 修改Pipeline的Kubernetes部署步骤，将Deployment的镜像直接指定为本次构建的唯一标签，而非beta浮标标签。K8s会因为标签唯一，强制拉取新镜像并触发滚动更新。
• 调整后的部署脚本示例：

  - step:
      name: Deploy to Beta Kubernetes
      script:
          - pipe: atlassian/kubectl-run:3.6.0
            variables:
                  KUBE_CONFIG: $KUBE_CONFIG
                  KUBECTL_COMMAND: >
                    set image deployment/backend-beta backend-beta=$REMOTE_IMAGE_TAG_W_BUILD &&
                    set image deployment/backend-queue-default-beta backend-queue-default-beta=$REMOTE_IMAGE_TAG_W_BUILD &&
                    set image deployment/backend-queue-notifications-beta backend-queue-notifications-beta=$REMOTE_IMAGE_TAG_W_BUILD
  

2. 替换固定sleep为镜像可用性校验 #

去掉sleep(60)，改为循环校验镜像是否可拉取，确保镜像同步完成后再部署：

# 在推送镜像的step末尾添加
- until docker pull $REMOTE_IMAGE_TAG_W_BUILD; do
    echo "Waiting for image to be available..."
    sleep 5
  done

3. 调整K8s镜像拉取策略（可选，配合浮标标签使用） #

若坚持使用beta浮标标签，需修改Deployment的镜像拉取策略为Always，强制Pod每次重启都拉取最新镜像：

# 在Deployment的Pod模板中添加
spec:
  containers:
  - name: backend-beta
    image: registry.digitalocean.com/xxxxx/k8s-xxxxx-backend:beta
    imagePullPolicy: Always

注：此方式会增加节点带宽消耗，可靠性不如唯一标签方案。

修改后的完整Pipeline示例 #

image: atlassian/default-image:3

options:
size: 1x

definitions:
services:
docker:
memory: 2048

pipelines:
branches:
beta-k8s:
- step:
    name: Build for Beta & Push Image To Beta DigitalOcean
    services:
        - docker
    script:
        - set -o pipefail
        - export VERSION_LABEL=`[[ $BITBUCKET_TAG ]] && echo "$BITBUCKET_TAG-${BITBUCKET_COMMIT::7}" || echo "$BITBUCKET_REPO_FULL_NAME-${BITBUCKET_COMMIT::7}"`
        - export VERSION_REF=`[[ $BITBUCKET_TAG ]] && echo "$BITBUCKET_TAG" || echo "$BITBUCKET_BRANCH"`
        - export BUILD_NUMBER=$BITBUCKET_BUILD_NUMBER
        - cd deployment/kubernetes && docker build --build-arg VERSION=$VERSION_LABEL -t xxxxx-backend:beta-build-$BUILD_NUMBER -t xxxxx-backend:beta -f ./Dockerfile ../.. | tee -a logs.txt 2>&1
        - export REMOTE_IMAGE_TAG="registry.digitalocean.com/xxxxx/k8s-xxxxx-backend:beta"
        - export REMOTE_IMAGE_TAG_W_BUILD="registry.digitalocean.com/xxxxx/k8s-xxxxx-backend:beta-build-$BUILD_NUMBER"
        - echo $REMOTE_IMAGE_TAG
        - echo $REMOTE_IMAGE_TAG_W_BUILD
        - docker login -u $DIGITALOCEAN_CONTAINER_REGISTRY_TOKEN -p $DIGITALOCEAN_CONTAINER_REGISTRY_TOKEN registry.digitalocean.com | tee -a logs.txt 2>&1
        - docker tag xxxxx-backend:beta $REMOTE_IMAGE_TAG | tee -a logs.txt 2>&1
        - docker tag xxxxx-backend:beta-build-$BUILD_NUMBER $REMOTE_IMAGE_TAG_W_BUILD | tee -a logs.txt 2>&1
        - docker push $REMOTE_IMAGE_TAG | tee -a logs.txt 2>&1
        - docker push $REMOTE_IMAGE_TAG_W_BUILD | tee -a logs.txt 2>&1
        # 替换固定sleep为镜像可用性校验
        - until docker pull $REMOTE_IMAGE_TAG_W_BUILD; do echo "Waiting for image sync..."; sleep 5; done
- step:
    name: Deploy to Beta Kubernetes
    script:
        - pipe: atlassian/kubectl-run:3.6.0
          variables:
                KUBE_CONFIG: $KUBE_CONFIG
                KUBECTL_COMMAND: >
                  set image deployment/backend-beta backend-beta=$REMOTE_IMAGE_TAG_W_BUILD &&
                  set image deployment/backend-queue-default-beta backend-queue-default-beta=$REMOTE_IMAGE_TAG_W_BUILD &&
                  set image deployment/backend-queue-notifications-beta backend-queue-notifications-beta=$REMOTE_IMAGE_TAG_W_BUILD

内容的提问来源于stack exchange，提问作者Čamo