azure azure-devops azure-pipelines azure-pipelines-yaml

Setting Variable in Azure Pipeline for use in other Jobs doesn’t work as expected

We are setting a variable in Azure Pipeline that we want to use in another job but the variable is not visible in the other job. The following is a simplified YML files we are using with the outputs from the pipeline.

File:BuildTemplate.yml

parameters:
- name: ShortVersion
  type: string
  default: ''

steps:
  - powershell: |
      $DATE = Get-Date
      $CUSTOM_VERSION = "${{ parameters.ShortVersion }}.$($DATE.ToString('yy'))$('{0:D3}' -f $Date.DayOfYear).$(CDP_DEFINITION_BUILD_COUNT)"
      if ('$(Build.SourceBranchName)' -eq "Release") {
        $NUGET_PACKAGE_VERSION = "$CUSTOM_VERSION"
      } else {
        $NUGET_PACKAGE_VERSION = "$CUSTOM_VERSION-$(Build.SourceBranchName)"
      }
    
    # Set output.
      Write-Host "##vso[task.setvariable variable=NUGET_PACKAGE_VERSION;isOutput=true]$NUGET_PACKAGE_VERSION"
      Write-Host "##vso[task.setvariable variable=CUSTOM_VERSION;]$CUSTOM_VERSION"
    name: SetNuGetPackageVersion
    displayName: 'Retrieve version'

Printing the variables set in BuildTemplate.yml, everything looks correct. We have another YML file where the BuildTemplate.yml is getting called from and it looks like something like below.

Based on this documentation for setting an output variable for use in future jobs, I would expect the linux_nuget_feed job should be able to see/set NuGetPackageVersion variable. However it doesn’t. What am I missing here? This doesn't seem to be a spacing issue in yml as far as I can tell.

- stage: linux
        dependsOn: []
        jobs:
          - job: linux_main
            pool:
              type: linux
            strategy:
              matrix:
                x64_mariner20:
                  LinuxContainerImage: "...."
                  Suffix: "_x64_mariner"
                  Triplet: "x64-linux"
                  ImageName: "Mariner2.0"
                x64_ubuntu2004:
                  LinuxContainerImage: " .... "
                  Suffix: "_x64_ubuntu"
                  Triplet: "x64-linux"
                  ImageName: "Ubuntu20.04"

            steps:

              - template: .pipelines/BuildTemplate.yml@self
                parameters:
                  ShortVersion: $(SHORT_VERSION)

              - task: PowerShell@2
                displayName: "Print NUGET_PACKAGE_VERSION"
                inputs:
                  targetType: "inline"
                  script: |
                    Write-Host "NUGET_PACKAGE_VERSION_0: $(SetNuGetPackageVersion.NUGET_PACKAGE_VERSION)"
                    Write-Host "##vso[task.setvariable variable=myOutputVar;isOutput=true]$(SetNuGetPackageVersion.NUGET_PACKAGE_VERSION)"
                name: passOutput

          - job: linux_nuget_feed
            dependsOn: linux_main
            pool:
              type: windows 
            strategy:
              matrix:
                x64_mariner20:
                  LinuxContainerImage: "...."
                  Suffix: "_x64_mariner"
                  Triplet: "x64-linux"
                  ImageName: "Mariner2.0"
                x64_ubuntu2004:
                  LinuxContainerImage: " .... "
                  Suffix: "_x64_ubuntu"
                  Triplet: "x64-linux"
                  ImageName: "Ubuntu20.04"
            variables:
              NuGetPackageVersion: $[ dependencies.linux_main.outputs['passOutput.myOutputVar'] ]

            steps:
              - task: PowerShell@2
                displayName: "Print NUGET_PACKAGE_VERSION"
                inputs:
                  targetType: "inline"
                  script: |
                    Write-Host "NUGET_PACKAGE_VERSION_0: $[ dependencies.linux_main.outputs['passOutput.myOutputVar'] ]""
                    Write-Host "NUGET_PACKAGE_VERSION_2: $(NuGetPackageVersion)"

Output of the Print NUGET_PACKAGE_VERSION from linux_main, this still looks correct.

Print NUGET_PACKAGE_VERSION (linux_build_container)

NUGET_PACKAGE_VERSION_0: 2.9.24213.125-Dev-2

Output of the Print NUGET_PACKAGE_VERSION from linux_nuget_feed, this is where things don't look right anymore.

Print NUGET_PACKAGE_VERSION (windows_build_container)

NUGET_PACKAGE_VERSION_0: $[ dependencies.linux_main.outputs['passOutput.myOutputVar'] ]"
Write-Host  NUGET_PACKAGE_VERSION_2:

Update: it looks like I missed " in my copy & paste to SoF but in the actual YML file I had the correct expression. Write-Host "NUGET_PACKAGE_VERSION_2: $(NuGetPackageVersion)" with the correct " at the end. However the problem still persists. i.e. it prints empty string.

I printed the dependencies passed linux_nuget_feed and I see something like below. The prefix here comes from the strategy > matrix portion of the YML that I initially omitted because I didn't think it was relevant here. I updated the YML file above.

checkDependencies:
  {
    linux_main:
      {
        outputs:
          {
            x64_ubuntu2004.agent.version: 3.242.1,
            x64_ubuntu2004.SetNuGetPackageVersion.NUGET_PACKAGE_VERSION: 2.9.24213.141-Dev-2,
            x64_ubuntu2004.passOutput.myOutputVar: 2.9.24213.141-Dev-2,
            x64_mariner20.SetNuGetPackageVersion.NUGET_PACKAGE_VERSION: 2.9.24213.141-Dev-2,
            x64_mariner20.passOutput.myOutputVar: 2.9.24213.141-Dev-2,

          },
        identifier: null,
        name: linux_main,
        attempt: 1,
        startTime: null,
        finishTime: null,
        state: NotStarted,
        result: Succeeded,
      },
  }

Solution

You have two outputs in linux_nuget_feed job:

$[ dependencies.linux_main.outputs['passOutput.myOutputVar'] is the runtime expression variable, which should be used in condition and expressions, while not output in task, it cannot be parsed. Remove this output in powershell task. Please check the usage for your reference.
The 2nd output format $(NuGetPackageVersion) is correct. But looks you loose " at the end.

Write-Host "NUGET_PACKAGE_VERSION_2: $(NuGetPackageVersion)"  # add " at the end

I copied your main yaml, and directly define the variable in 1st job as value is correctly set in template:

trigger: none

stages:
  - stage: linux
    dependsOn: []
    jobs:
      - job: linux_main
        pool:
          vmImage: Ubuntu-latest
        
        variables:
          SetNuGetPackageVersion.NUGET_PACKAGE_VERSION: 2.9.24213.125-Dev-2
          
        steps:

          # - template: .pipelines/BuildTemplate.yml@self
          #   parameters:
          #     ShortVersion: $(SHORT_VERSION)

          - task: PowerShell@2
            displayName: "Print NUGET_PACKAGE_VERSION"
            inputs:
              targetType: "inline"
              script: |
                Write-Host "NUGET_PACKAGE_VERSION_0: $(SetNuGetPackageVersion.NUGET_PACKAGE_VERSION)"
                Write-Host "##vso[task.setvariable variable=myOutputVar;isOutput=true]$(SetNuGetPackageVersion.NUGET_PACKAGE_VERSION)"
            name: passOutput

      - job: linux_nuget_feed
        dependsOn: linux_main
        pool:
          vmImage: Windows-latest
        variables:
          NuGetPackageVersion: $[ dependencies.linux_main.outputs['passOutput.myOutputVar'] ]

        steps:
          - task: PowerShell@2
            displayName: "Print NUGET_PACKAGE_VERSION"
            inputs:
              targetType: "inline"
              script: |
                Write-Host "NUGET_PACKAGE_VERSION_2: $(NuGetPackageVersion)"

I can output the variable in linux_nuget_feed job as below:

You can follow above fixed yaml for a check.

Edit:

Since you have matrix defined in the 1st linux_main job, for each occurrence in the matrix, a copy of the job is generated.

Hence, to use the output variable in the next job linux_nuget_feed, you need to add the matrix name(x64_mariner20 or x64_ubuntu2004) in the expression to specify which output variable invoked , please fixe as below in 2nd job:

NuGetPackageVersion: $[ dependencies.linux_main.outputs['x64_mariner20.passOutput.myOutputVar'] ]