Azure YAML Get variable from a job run in a previous stage

Question

I am creating YAML pipeline in Azure DevOps that consists of two stages.

The first stage (Prerequisites) is responsible for reading the git commit and creates a comma separated variable containing the list of services that has been affected by the commit.

The second stage (Build) is responsible for building and unit testing the project. This Stage consists of many templates, one for each Service. In the template script, the job will check if the relevant Service in in the variable created in the previous stage. If the job finds the Service it will continue to build and test the service. However if it cannot find the service, it will skip that job.

Run.yml:

stages:
- stage: Prerequisites
  jobs:
  - job: SetBuildQueue
    steps:
    - task: powershell@2
      name: SetBuildQueue
      displayName: 'Set.Build.Queue'
      inputs:
        targetType: inline
        script: |
            ## ... PowerShell script to get changes - working as expected
            Write-Host "Build Queue Auto: $global:buildQueueVariable"
            Write-Host "##vso[task.setvariable variable=buildQueue;isOutput=true]$global:buildQueueVariable"

- stage: Build
  jobs:
  - job: StageInitialization

  - template: Build.yml
    parameters:
      projectName: Service001
      projectLocation: src/Service001

  - template: Build.yml
    parameters:
      projectName: Service002
      projectLocation: src/Service002

Build.yml:

parameters:
  projectName: ''
  projectLocation: ''

jobs:
- job:
  displayName: '${{ parameters.projectName }} - Build'
  dependsOn: SetBuildQueue
  continueOnError: true
  condition: and(succeeded(), contains(dependencies.SetBuildQueue.outputs['SetBuildQueue.buildQueue'], '${{ parameters.projectName }}'))
  steps:
  - task: NuGetToolInstaller@1
    displayName: 'Install Nuget'

Issue:

When the first stages runs it will create a variable called buildQueue which is populated as seen in the console output of the PowerShell script task:

Service001 Changed
Build Queue Auto: Service001;

However when it gets to stage two and it tries to run the build template, when it checks the conditions it returns the following output:

Started: Today at 12:05 PM
Duration: 16m 7s
Evaluating: and(succeeded(), contains(dependencies['SetBuildQueue']['outputs']['SetBuildQueue.buildQueue'], 'STARS.API.Customer.Assessment'))
Expanded: and(True, contains(Null, 'service001'))
Result: False

So my question is how do I set the dependsOn and condition to get the information from the previous stage?

Answer 1

It because you want to access the variable in a different stage from where you defined them. currently, it's impossible, each stage it's a new instance of a fresh agent.

In this blog you can find a workaround that involves writing the variable to disk and then passing it as a file, leveraging pipeline artifacts.

To pass the variable FOO from a job to another one in a different stage:

1. Create a folder that will contain all variables you want to pass; any folder could work, but something like mkdir -p $(Pipeline.Workspace)/variables might be a good idea.
2. Write the contents of the variable to a file, for example echo "$FOO" > $(Pipeline.Workspace)/variables/FOO. Even though the name could be anything you’d like, giving the file the same name as the variable might be a good idea.
3. Publish the $(Pipeline.Workspace)/variables folder as a pipeline artifact named variables
4. In the second stage, download the variables pipeline artifact
5. Read each file into a variable, for example FOO=$(cat $(Pipeline.Workspace)/variables/FOO)
6. Expose the variable in the current job, just like we did in the first example: echo "##vso[task.setvariable variable=FOO]$FOO"
7. You can then access the variable by expanding it within Azure Pipelines ($(FOO)) or use it as an environmental variable inside a bash script ($FOO).