Azure Pipeline跨Job传递输出变量失败问题排查

问题描述 #

我们在Azure Pipeline中设置了输出变量，希望在另一个Job中使用，但目标Job无法获取到该变量。相关配置及现象如下：

1. 模板文件（BuildTemplate.yml） #

parameters:
- name: ShortVersion
  type: string
  default: ''

steps:
  - powershell: |
      $DATE = Get-Date
      $CUSTOM_VERSION = "${{ parameters.ShortVersion }}.$($DATE.ToString('yy'))$('{0:D3}' -f $Date.DayOfYear).$(CDP_DEFINITION_BUILD_COUNT)"
      if ('$(Build.SourceBranchName)' -eq "Release") {
        $NUGET_PACKAGE_VERSION = "$CUSTOM_VERSION"
      } else {
        $NUGET_PACKAGE_VERSION = "$CUSTOM_VERSION-$(Build.SourceBranchName)"
      }
    
      # Set output.
      Write-Host "##vso[task.setvariable variable=NUGET_PACKAGE_VERSION;isOutput=true]$NUGET_PACKAGE_VERSION"
      Write-Host "##vso[task.setvariable variable=CUSTOM_VERSION;]$CUSTOM_VERSION"
    name: SetNuGetPackageVersion
    displayName: 'Retrieve version'

2. 主流水线配置 #

- stage: linux
  dependsOn: []
  jobs:
    - job: linux_main
      pool:
        type: linux
      strategy:
        matrix:
          x64_mariner20:
            LinuxContainerImage: "...."
            Suffix: "_x64_mariner"
            Triplet: "x64-linux"
            ImageName: "Mariner2.0"
          x64_ubuntu2004:
            LinuxContainerImage: " .... "
            Suffix: "_x64_ubuntu"
            Triplet: "x64-linux"
            ImageName: "Ubuntu20.04"

      steps:
        - template: .pipelines/BuildTemplate.yml@self
          parameters:
            ShortVersion: $(SHORT_VERSION)

        - task: PowerShell@2
          displayName: "Print NUGET_PACKAGE_VERSION"
          inputs:
            targetType: "inline"
            script: |
              Write-Host "NUGET_PACKAGE_VERSION_0: $(SetNuGetPackageVersion.NUGET_PACKAGE_VERSION)"
              Write-Host "##vso[task.setvariable variable=myOutputVar;isOutput=true]$(SetNuGetPackageVersion.NUGET_PACKAGE_VERSION)"
          name: passOutput

    - job: linux_nuget_feed
      dependsOn: linux_main
      pool:
        type: windows 
      strategy:
        matrix:
          x64_mariner20:
            LinuxContainerImage: "...."
            Suffix: "_x64_mariner"
            Triplet: "x64-linux"
            ImageName: "Mariner2.0"
          x64_ubuntu2004:
            LinuxContainerImage: " .... "
            Suffix: "_x64_ubuntu"
            Triplet: "x64-linux"
            ImageName: "Ubuntu20.04"
      variables:
        NuGetPackageVersion: $[ dependencies.linux_main.outputs['passOutput.myOutputVar'] ]

      steps:
        - task: PowerShell@2
          displayName: "Print NUGET_PACKAGE_VERSION"
          inputs:
            targetType: "inline"
            script: |
              Write-Host "NUGET_PACKAGE_VERSION_0: $[ dependencies.linux_main.outputs['passOutput.myOutputVar'] ]"
              Write-Host "NUGET_PACKAGE_VERSION_2: $(NuGetPackageVersion)"

3. 现象 #

• linux_main中打印变量正常：

Print NUGET_PACKAGE_VERSION (linux_build_container)

NUGET_PACKAGE_VERSION_0: 2.9.24213.125-Dev-2

• linux_nuget_feed中打印变量为空：

Print NUGET_PACKAGE_VERSION (windows_build_container)

NUGET_PACKAGE_VERSION_0: $[ dependencies.linux_main.outputs['passOutput.myOutputVar'] ]
Write-Host  NUGET_PACKAGE_VERSION_2:

4. 依赖输出排查 #

打印依赖发现输出变量带有矩阵策略的前缀：

checkDependencies:
  {
    linux_main:
      {
        outputs:
          {
            x64_ubuntu2004.agent.version: 3.242.1,
            x64_ubuntu2004.SetNuGetPackageVersion.NUGET_PACKAGE_VERSION: 2.9.24213.141-Dev-2,
            x64_ubuntu2004.passOutput.myOutputVar: 2.9.24213.141-Dev-2,
            x64_mariner20.SetNuGetPackageVersion.NUGET_PACKAGE_VERSION: 2.9.24213.141-Dev-2,
            x64_mariner20.passOutput.myOutputVar: 2.9.24213.141-Dev-2,

          },
        identifier: null,
        name: linux_main,
        attempt: 1,
        startTime: null,
        finishTime: null,
        state: NotStarted,
        result: Succeeded,
      },
  }

原因分析 #

linux_main Job使用了matrix策略，每个矩阵实例的输出变量都会自动带上对应的矩阵键（如x64_ubuntu2004.、x64_mariner20.）作为前缀。当前linux_nuget_feed中直接引用dependencies.linux_main.outputs['passOutput.myOutputVar']时，没有指定对应的矩阵实例，因此无法匹配到正确的变量值。

同时linux_nuget_feed也使用了和linux_main完全相同的matrix策略，可以利用当前Job的矩阵实例名称来匹配对应的前置Job输出。

解决方案 #

修改linux_nuget_feed中的变量定义，使用${{ variables['Agent.JobName'] }}动态获取当前矩阵实例的名称，拼接成正确的输出变量路径：

- job: linux_nuget_feed
  dependsOn: linux_main
  pool:
    type: windows 
  strategy:
    matrix:
      x64_mariner20:
        LinuxContainerImage: "...."
        Suffix: "_x64_mariner"
        Triplet: "x64-linux"
        ImageName: "Mariner2.0"
      x64_ubuntu2004:
        LinuxContainerImage: " .... "
        Suffix: "_x64_ubuntu"
        Triplet: "x64-linux"
        ImageName: "Ubuntu20.04"
  variables:
    # 动态匹配当前矩阵实例对应的前置Job输出变量
    NuGetPackageVersion: $[ dependencies.linux_main.outputs['${{ variables['Agent.JobName'] }}.passOutput.myOutputVar'] ]

  steps:
    - task: PowerShell@2
      displayName: "Print NUGET_PACKAGE_VERSION"
      inputs:
        targetType: "inline"
        script: |
          Write-Host "NUGET_PACKAGE_VERSION_0: $[ dependencies.linux_main.outputs['${{ variables['Agent.JobName'] }}.passOutput.myOutputVar'] ]"
          Write-Host "NUGET_PACKAGE_VERSION_2: $(NuGetPackageVersion)"

如果不需要和前置Job的矩阵一一对应，而是想固定获取某个矩阵实例的输出（比如只取x64_ubuntu2004的变量），可以直接指定矩阵键：

NuGetPackageVersion: $[ dependencies.linux_main.outputs['x64_ubuntu2004.passOutput.myOutputVar'] ]

验证 #

修改后重新运行流水线，linux_nuget_feed中的PowerShell任务应该能正常打印出NUGET_PACKAGE_VERSION的值，和linux_main中对应矩阵实例的输出一致。

内容的提问来源于stack exchange，提问作者AR5HAM