使用CircleCI部署Docker至Kubernetes时Rollout超时失败的修复咨询

使用CircleCI结合Kubernetes部署Docker镜像时，执行命令./bin/kubectl rollout status deployments/prj5-deploy --timeout=2700s约20分钟后出现两类错误：

• CircleCI提示：Too long with no output (exceeded 40m0s): context deadline exceeded
• Kubernetes返回：error: deployment "****-deploy" exceeded its progress deadline

相关配置文件如下：

部署脚本 deploy-application.yml #

- name: "Deploy application"
  hosts: web[0]
  user: ubuntu
  gather_facts: false
  become: yes
  tasks:
    - name: build
      shell: "./bin/kubectl set image deployments/prj5-deploy prj5-app=shalltearbloodfallen01/prj5-deploy:master"
      args:
        chdir: $HOME  

    - name: if successful
      shell: "./bin/kubectl rollout status deployments/prj5-deploy --timeout=2700s"
      args:
        chdir: $HOME

Deployment配置 prj5-deploy.yml #

apiVersion: apps/v1
kind: Deployment
metadata:
  name: prj5-deploy
  labels:
    app: prj5-app
spec:
  paused: false
  replicas: 4
  selector:
    matchLabels:
      app: prj5-app
  strategy:
    type: RollingUpdate
    rollingUpdate:
        maxSurge: 0
        maxUnavailable: 1
  template:
    metadata:
      labels:
        app: prj5-app
    spec:
      containers:
        - name: prj5-app
          image: shalltearbloodfallen01/prj5-deploy
          ports:
            - containerPort: 80

CircleCI配置 config.yml #

deploy-docker:
    docker:
      - image: python:3.7-alpine3.11
    steps:
      - checkout
      - add_ssh_keys:
          fingerprints: ["xxxxxxxxxxxxxxxxxxxxxxxxxx"]
      - attach_workspace:
          at: ~/
      - run:
          name: Install dependencies
          command: |
            apk add --update ansible
      - run:
          name: deploy image docker
          no_output_timeout: 40m
          command: |
            cd .circleci/ansible
            cat inventory.txt
            ansible-playbook -i inventory.txt deploy-application.yml

1. 延长Kubernetes Deployment进度超时时间 #

Kubernetes默认Deployment进度超时为10分钟，部署20分钟未完成会触发超时。在Deployment的spec中添加progressDeadlineSeconds参数，延长超时窗口：

apiVersion: apps/v1
kind: Deployment
metadata:
  name: prj5-deploy
  labels:
    app: prj5-app
spec:
  paused: false
  replicas: 4
  progressDeadlineSeconds: 3600  # 设置为1小时，根据实际需求调整
  selector:
    matchLabels:
      app: prj5-app
  # 其余配置保持不变

2. 优化滚动更新策略提速 #

当前maxSurge: 0+maxUnavailable:1的配置限制了每次仅能替换1个Pod，4个Pod需依次更新，速度极慢。调整参数允许并行更新：

rollingUpdate:
    maxSurge: 1  # 允许额外创建1个Pod，实现并行更新
    maxUnavailable: 1  # 最多允许1个Pod不可用，保证服务可用性

若集群资源充足，可进一步将maxSurge设为2，大幅缩短部署时间。

3. 解决CircleCI无输出超时问题 #

kubectl rollout status在部署过程中可能长时间无输出，触发CircleCI的no_output_timeout。修改命令定期输出状态，同时延长CircleCI超时时间：

修改Ansible中的rollout任务 #

- name: if successful
  shell: |
    ./bin/kubectl rollout status deployments/prj5-deploy --timeout=3600s &
    ROLLOUT_PID=$!
    # 每隔60秒输出一次进度提示，避免无输出超时
    while kill -0 $ROLLOUT_PID 2>/dev/null; do
      echo "Deployment in progress, waiting for pods to be ready..."
      sleep 60
    done
    wait $ROLLOUT_PID
  args:
    chdir: $HOME

延长CircleCI的无输出超时 #

- run:
    name: deploy image docker
    no_output_timeout: 60m  # 延长至1小时，匹配Deployment超时时间
    command: |
      cd .circleci/ansible
      cat inventory.txt
      ansible-playbook -i inventory.txt deploy-application.yml

4. 排查Pod启动慢的根源 #

部署耗时久的核心可能是Pod启动效率低，需针对性优化：

• 精简Docker镜像：使用多阶段构建、Alpine等轻量基础镜像，减少镜像体积
• 配置镜像缓存：使用私有镜像仓库或集群本地缓存，加快镜像拉取速度
• 检查应用启动日志：确认是否存在数据库迁移、大量配置加载等耗时初始化步骤
• 添加就绪/存活探针：配置readinessProbe和livenessProbe，让Kubernetes准确判断Pod状态，避免无效等待

内容的提问来源于stack exchange，提问作者alex