use output variables from another stage in a different stage

Question

I have the next code snippets:

- task: AzureCLI@2
  name: A
  displayName: "Run tf ${{ parameters.action }}"
  inputs:
    azureSubscription: '***-${{ parameters.env }}' 
    scriptType: 'bash'
    scriptLocation: 'inlineScript'
    inlineScript: |
      if [ ${{ parameters.action }} = "plan" ]; then
          echo "Running terraform plan..."
          timeout -s SIGINT 58m terraform plan out=terraform.plan
          changes=$(terraform show -no-color terraform.plan | grep -E "No changes.")
          echo "Checking for Changes variable [$changes]"
          if [ -n "$changes" ]; then
            echo "No changes detected. Exiting without further actions."
            echo '##vso[task.setvariable variable=PlanChanges; isOutput=true]'notdetected
          else
            echo '##vso[task.setvariable variable=PlanChanges; isOutput=true]'changesdetected
          fi
      else
          echo "Running terraform apply..."
          timeout -s SIGINT 58m terraform apply input=false auto-approve
      fi
    addSpnToEnvironment: true
    workingDirectory: '${{ parameters.workingDirectory }}'    

These tasks are part of a template called script.yml and it is used in another template called stages.yml following the next structure in the next comment:

parameters:
- name: workingDirectory
  type: string
- name: environments
  type: object
  default:
  - dev
  - qa


stages:
- ${{ each env in parameters.environments }}:
    - stage: Plan_${{ env }}
      condition: and(succeeded(), ne(variables['Build.Reason'], 'PullRequest'))
      jobs:
        - job: Plan
          steps:
          - template: script.yml
            parameters:
              env: ${{ env }}
              action: plan
              workingDirectory: '${{ parameters.workingDirectory }}'

    - stage: Apply_${{ env }}
      dependsOn:
        - Plan_${{ env }}
      jobs: 
      - deployment: 
        displayName: Deploy
        environment: ${{ env }}
        strategy:
          runOnce:
            deploy:
              steps:            
              - template: script.yml
                parameters:
                  env: ${{ env }}
                  action: apply
                  workingDirectory: '${{ parameters.workingDirectory }}'

Now, I have isOutput=true option for (echo '##vso[task.setvariable variable=PlanChanges; isOutput=true]'notdetected), I need to consume this output variable in my Apply stage which is a different one. I need to set a condition based on this output for the Apply Stage to be skipped/to run only the variable output is "changesdetected"... but I tried first to consume that variable to see if works.

I have tried referencing this variable in the stages.yml: adding variables: varPlan: $[ stageDependencies.Plan_${{ env }}.Plan.outputs['PlanOverview.PlanChanges'] ] or varPlan: $[ stageDependencies.Plan_${{ env }}.outputs['Plan.PlanOverview.PlanChanges'] ] when I run the script echo $(varPlan) not showing the output of variable.

According to: https://learn.microsoft.com/en-us/azure/devops/pipelines/process/expressions?view=azure-devops&branch=pr-en-us-1603#job-to-job-dependencies-across-stages:~:text=Dependency%20syntax%20overview

how to reference that variable output to my condition in the apply stage?

Tried like these: varPlan: $[ stageDependencies.Plan_${{ env }}.Plan.outputs['[email protected]'] ]

varPlan: $[ stageDependencies.Plan_${{ env }}.Plan.outputs['A.PlanChanges'] ]

varPlan: $[ dependencies.Plan_${{ env }}.outputs['[email protected]'] ]

varPlan: $[ stageDependencies.Plan_${{ env }}.outputs['Plan.A.PlanChanges'] ]

varPlan: $[ stageDependencies.Plan_${{ env }}.Plan.A.outputs['PlanChanges'] ]

varPlan: $[ stageDependencies.Plan_${{ env }}.Plan.outputs['PlanOverview.PlanChanges'] ]

varPlan: $[ stageDependencies.Plan_${{ env }}.PlanOverview.outputs['PlanChanges'] ]

varPlan: $[ stageDependencies.Plan_${{ env }}.Plan.outputs['PlanOverview.PlanChanges'] ]

Answer 1

On the stage-level of all the "Apply_xxx" stages, you can set the condition like as below.

- ${{ each env in parameters.environments }}:
  - stage: Plan_${{ env }}
    . . .
  
  - stage: Apply_${{ env }}
    dependsOn: Plan_${{ env }}
    condition: and(succeeded(), eq(stageDependencies.Plan_${{ env }}.outputs['Plan.A.PlanChanges'], 'changesdetected'))

With this condition, the "Apply_xxx" stages will run when the value of output variable "PlanChanges" is "changesdetected".

---

In addition, another important thing you need to know is that based on the current definition in your main YAML (stages.yml), all the stages will run in a single line in sequence based on their order of defining in the YAML file. So, they will look like as below in a pipeline run.

In this situation, a stage will be skipped if any of the previous stages is not succeeded. For example, on above image, in the expected result, the "Apply_prod" stage should run but it was skipped due to the previous "Apply_qa" stage was skipped.

For your case, the ideal results should be:

• "Plan_dev" and "Apply_dev" are in a separate line, and "Apply_dev" only depends on "Plan_dev".
• "Plan_qa" and "Apply_qa" are in a separate line, and "Apply_qa" only depends on "Plan_qa".
• "Plan_prod" and "Apply_prod" are in a separate line, and "Apply_prod" only depends on "Plan_prod".

To reach this, you can update the main YAML (stages.yml) like as below:

1. Add a "main" stage as the parent node of all the separate lines. In this "main" stage, you can let it do nothing. So, it will be always succeeded.
2. Set the "Plan_xxx" stage to depend on the "main" stage.
3. Set the "Apply_xxx" stage to depend on the "Plan_xxx" stage.

stages:
- stage: main
  jobs:
  - job: main
    steps:
    - checkout: none

- ${{ each env in parameters.environments }}:
  - stage: Plan_${{ env }}
    dependsOn: main
    condition: and(succeeded(), ne(variables['Build.Reason'], 'PullRequest'))
    jobs:
    . . .
  
  - stage: Apply_${{ env }}
    dependsOn: Plan_${{ env }}
    condition: and(succeeded(), eq(stageDependencies.Plan_${{ env }}.outputs['Plan.A.PlanChanges'], 'changesdetected'))
    jobs:
    . . .

---

EDIT:

• For the question in your first reply blow:

  If I did not misnderstand your demands, when the value of output variable (PlanChanges) generated from 'Plan_${{ env }}' stage is 'changesdetected', then the corresponding 'Apply_${{ env }}' stage should run and not skip, and when the output value is 'notdetected', the 'Apply_${{ env }}' stage should skip.

  So, if 'Plan_prod' stage outputs 'changesdetected', the 'Apply_prod' stage should run. On the first pipeline image I posted above, the 'Plan_prod' stage was outputting 'changesdetected', the 'Apply_prod' stage should not skip.

• For the question in your second reply below:

  You seem have wrong syntax when use the setvariable command to set the output variable using Bash.

  Change the command line like as the following:

  echo "##vso[task.setvariable variable=PlanChanges;isoutput=true]notdetected"
  echo "##vso[task.setvariable variable=PlanChanges;isoutput=true]changesdetected"
  

---

EDIT_2:

Below I will share you with a sample that I attempted on my side. In this sample, I set the "Apply_qa" stage will be skipped based on the condition below.

condition: and(succeeded(), eq(stageDependencies.Plan_${{ env }}.outputs['Plan.A.PlanChanges'], 'changesdetected'))

You can reference this sample to check and update the code in your YAML files.

• The main YAML file: azure-pipelines.yml

parameters:
- name: workingDirectory
  type: string
- name: environments
  type: object
  default:
  - dev
  - qa
  - prod

stages:
- stage: main
  jobs:
  - job: main
    steps:
    - checkout: none

- ${{ each env in parameters.environments }}:
  - stage: Plan_${{ env }}
    dependsOn: main
    condition: and(succeeded(), ne(variables['Build.Reason'], 'PullRequest'))
    jobs:
    - job: Plan
      steps:
      - template: script.yml
        parameters:
          env: ${{ env }}
          action: plan
          workingDirectory: '${{ parameters.workingDirectory }}'
  
  - stage: Apply_${{ env }}
    dependsOn: Plan_${{ env }}
    condition: and(succeeded(), eq(stageDependencies.Plan_${{ env }}.outputs['Plan.A.PlanChanges'], 'changesdetected'))
    jobs:
    - deployment: Deploy
      environment: ${{ env }}
      variables:
      # The variable can be used by the steps (include script.yml) within this job. 
        varPlanChanges: $[ stageDependencies.Plan_${{ env }}.Plan.outputs['A.PlanChanges'] ]
      strategy:
        runOnce:
          deploy:
            steps:
            - template: script.yml
              parameters:
                env: ${{ env }}
                action: apply
                workingDirectory: '${{ parameters.workingDirectory }}'

• The template YAML file: script.yml

steps:
- task: Bash@3
  name: A
  displayName: 'Set Output'
  inputs:
    targetType: inline
    script: |
      if [ ${{ parameters.action }} = "plan" ]; then
        echo "The action is ${{ parameters.action }}."
        echo "The environment is ${{ parameters.env }}."
        if [ ${{ parameters.env }} = "qa" ]; then
          echo "##vso[task.setvariable variable=PlanChanges;isoutput=true]notdetected"
        else
          echo "##vso[task.setvariable variable=PlanChanges;isoutput=true]changesdetected"
        fi
      else
        echo "The action is not plan. It is ${{ parameters.action }}."
        echo "The environment is ${{ parameters.env }}."
      fi

- ${{ if eq(parameters.action, 'plan') }}:
  - task: Bash@3
    displayName: 'Variables for plan action'
    inputs:
      targetType: inline
      script: |
        echo "env = ${{ parameters.env }}"
        echo "action = ${{ parameters.action }}"
        echo "workingDirectory = ${{ parameters.workingDirectory }}"
        echo "PlanChanges = $(A.PlanChanges)"

- ${{ if eq(parameters.action, 'apply') }}:
  - task: Bash@3
    displayName: 'Variables for apply action'
    inputs:
      targetType: inline
      script: |
        echo "env = ${{ parameters.env }}"
        echo "action = ${{ parameters.action }}"
        echo "workingDirectory = ${{ parameters.workingDirectory }}"
        echo "varPlanChanges = $(varPlanChanges)"

• The result of running the pipeline, see the second image I posted above.

---