Azure DevOps中Stage1变量导出复用至Stage2失败问题

场景描述 #

尝试在Azure DevOps流水线中，将publish_service_names阶段获取的服务名称列表传递到publish_docker_images阶段，循环调用模板为每个服务构建推送Docker镜像，但两种尝试方式均失败：

1. 直接在模板循环表达式中使用stageDependencies变量
2. 将变量赋值后再用于循环

原流水线代码 #

- stage: publish_service_names
  dependsOn: build_code
  jobs:
    - job: get_services
      pool:
        ${{ if eq(parameters.useCustomAgentPool, true) }}:
          name: ${{ parameters.customAgentPoolName }}
        ${{ if eq(parameters.useCustomAgentPool, false) }}:
          vmImage: ubuntu-latest
      steps:
        - bash: |
            services=$(cat $(Pipeline.Workspace)/variables/services.txt)
            echo "service List : $services"
            echo "##vso[task.setvariable variable=serviceName;]$services"
          name: getservices
- ${{ if and(eq(parameters.buildDockerImages, true), eq(parameters.buildSingleServiceDockerImages, true)) }}:
    - stage: publish_docker_images
      dependsOn: publish_service_names
      variables:
        services: $[stageDependencies.publish_service_names.get_services.outputs['getservices.serviceName']]
      jobs:
        - ${{ each service in split(variables.services, ',') }}:
          - template: ./build-push-single-docker.yaml
            parameters:
              acrServiceConnection: ${{ parameters.acrServiceConnection}}
              serviceName: "${{ service }}"
              friendlyName: "${{ service }}"
              repositoryPathPrefix: ${{ parameters.repositoryPathPrefix }}

模板文件build-push-single-docker.yaml代码 #

jobs:
  - job: publish_${{ parameters.friendlyName }}
    pool:
      ${{ if eq(parameters.useCustomAgentPool, true) }}:
        name: ${{ parameters.customAgentPoolName }}
      ${{ if eq(parameters.useCustomAgentPool, false) }}:
        vmImage: ubuntu-latest
    steps:
      - bash: |
          echo "##vso[task.setvariable variable=buildArtifact;]ailens_services_supplemented_with_client_package"
        condition: eq('${{ parameters.addClientToSingleboxAndGUI }}', true)
        displayName: "Set artifact name"

问题根源 #

Azure DevOps的模板表达式${{ }}属于编译时执行，会在流水线运行前解析；而stageDependencies获取的变量是运行时变量，只有在流水线运行阶段才会赋值。因此编译时循环${{ each }}无法读取到运行时才生成的services变量，导致循环不生效。

解决方法 #

通过中转Job将运行时变量转换为可用于动态作业生成的格式，再利用矩阵策略实现循环调用模板：

修改后的publish_docker_images阶段代码 #

- ${{ if and(eq(parameters.buildDockerImages, true), eq(parameters.buildSingleServiceDockerImages, true)) }}:
  - stage: publish_docker_images
    dependsOn: publish_service_names
    variables:
      services: $[stageDependencies.publish_service_names.get_services.outputs['getservices.serviceName']]
    jobs:
      # 中转Job：将逗号分隔的服务名转换为JSON数组，输出为运行时变量
      - job: generate_docker_jobs
        pool:
          ${{ if eq(parameters.useCustomAgentPool, true) }}:
            name: ${{ parameters.customAgentPoolName }}
          ${{ if eq(parameters.useCustomAgentPool, false) }}:
            vmImage: ubuntu-latest
        steps:
          - bash: |
              # 将逗号分隔的字符串转为JSON数组格式
              services_array=$(echo '["'"$(echo $services | sed 's/,/","/g")"'"]')
              echo "##vso[task.setvariable variable=servicesArray;isOutput=true]$services_array"
            name: generate_vars
      # 利用矩阵策略动态生成每个服务的构建作业
      - job: publish_docker
        dependsOn: generate_docker_jobs
        pool:
          ${{ if eq(parameters.useCustomAgentPool, true) }}:
            name: ${{ parameters.customAgentPoolName }}
          ${{ if eq(parameters.useCustomAgentPool, false) }}:
            vmImage: ubuntu-latest
        variables:
          # 获取中转Job输出的JSON数组变量
          servicesArray: $[dependencies.generate_docker_jobs.outputs['generate_vars.servicesArray']]
        strategy:
          matrix:
            # 编译时解析JSON数组，生成每个服务的矩阵项
            ${{ each service in convertJson(variables.servicesArray) }}:
              ${{ service }}:
                serviceName: ${{ service }}
        steps:
          - template: ./build-push-single-docker.yaml
            parameters:
              acrServiceConnection: ${{ parameters.acrServiceConnection}}
              serviceName: $(serviceName)
              friendlyName: $(serviceName)
              repositoryPathPrefix: ${{ parameters.repositoryPathPrefix }}
              useCustomAgentPool: ${{ parameters.useCustomAgentPool }}
              customAgentPoolName: ${{ parameters.customAgentPoolName }}
              addClientToSingleboxAndGUI: ${{ parameters.addClientToSingleboxAndGUI }}

说明 #

1. 中转Job：负责将运行时获取的逗号分隔服务名字符串转换为JSON数组，并标记为输出变量，供后续Job使用。
2. 矩阵策略：通过convertJson函数将JSON数组转换为编译时可遍历的集合，动态生成每个服务对应的矩阵项，从而实现循环调用模板的效果。
3. 参数传递：确保模板所需的所有参数（如useCustomAgentPool等）都完整传递，避免模板执行出错。

内容的提问来源于stack exchange，提问作者Venkat Reddy