how to use an output variable as field value of another task in the same job?

Question

I'm having issue with the last task in my pipeline, in the the value of kubernetesServiceConnection, it seems it is reading $(GetServiceConnection.serviceconnection) as a literal value. And here's the error message I'm getting:

The pipeline is not valid. Job ProcessApp_FileGen: Step KubernetesManifest input kubernetesServiceEndpoint references service connection $(GetServiceConnection.serviceconnection) which could not be found. The service connection does not exist, has been disabled or has not been authorized for use

but if I remove the last 2 tasks, the 2nd task can echo the actual value of $(GetServiceConnection.serviceconnection) properly. I also made sure that the service connections can be accessed by all of the pipelines in the project. I even tried enclosing it with single quotes and double quotes, but I'm still receiving the same error.

stages:
- stage: ProcessApps
  jobs:
  - job: DummyJob

  - ${{ each app in parameters.apps }}:
    - job: ProcessApp_${{ app.application }}
      displayName: "Processing App '${{ app.application }}'"      
      steps:

      - ${{ each overlay in app.environments }}:
        - task: Bash@3
          name: GetServiceConnection
          displayName: Get Service Connection for '${{ overlay }}'
          inputs:
            targetType: inline
            script: |
              # simplified code
              service_connection="sc-aks-dev"

              echo "##vso[task.setvariable variable=serviceconnection;isOutput=true]${service_connection}"              

        - task: Bash@3
          displayName: Processing Overlay '${{ overlay }}'
          inputs:            
            targetType: inline
            script: |
              echo $(GetServiceConnection.serviceconnection)

        - task: KubernetesManifest@1
          name: bake_${{ app.application }}_${{ overlay }}
          displayName: Baking ${{ app.application }} - ${{ overlay }}
          inputs:
            action: bake
            renderType: kustomize
            kustomizationPath: '$(System.DefaultWorkingDirectory)/01_applications/${{ app.application }}/overlays/${{ overlay }}'

        - task: KubernetesManifest@1
          displayName: Deploying
          inputs:
            kubernetesServiceConnection: $(GetServiceConnection.serviceconnection)
            manifests: $(bake_${{ app.application }}_${{ overlay }}.manifestsBundle)

but if I change it to ${{ GetServiceConnection.serviceconnection }}, I'm getting this error when I run the pipeline:

Unrecognized value: 'GetServiceConnection'. Located at position 1 within expression: GetServiceConnection.serviceconnection

Answer 1

The error message you’re seeing:

The pipeline is not valid. Job ProcessApp_FileGen: Step KubernetesManifest input kubernetesServiceEndpoint references service connection $(GetServiceConnection.serviceconnection) which could not be found. The service connection does not exist, has been disabled or has not been authorized for use

indicates that the kubernetesServiceConnection field is taking the variable reference literally, rather than expanding it to the value set in the GetServiceConnection task. According to the table in the offical document Runtime expression syntax, this is how the $(GetServiceConnection.serviceconnection) is rendered when it is not found.

The issue arises because the variable is not available at the time the service connection is being resolved, which happens before the pipeline starts executing its tasks.

In other words, we cannot use variables that are set during the pipeline execution using the task.setvariable command in the kubernetesServiceConnection field. The service connections are resolved before the pipeline starts executing, so they cannot access variables that are set during the pipeline run.

Workaround：

I noticed that the question is a following question of another question you have asked. To workaround the issue, you can also use the same way in that question's answer to pass the service connection name to the second pipeline as parameters.

Assume that the format of the service connection you need is the prefix sc-aks(or can be different for different app), plus the name of each environment, such as dev, which becomes sc-aks-dev.

Pipeline One:

stages:
  - stage: Initialize
    displayName: Initialize
    jobs:
    - job: AppsList
      steps:
      - task: Bash@3
        name: Applications
        inputs:
          targetType: inline
          script: |
   
            applications=(
              '{"application": "app1","serviceconnectionprefix": "sc-aks", "environments": ["dev", "test", "production"]},'
              '{"application": "app2", "serviceconnectionprefix": "sc-aks","environments": ["dev", "staging"]}'
            )
            echo "##vso[task.setvariable variable=apps;isOutput=true]${applications[@]}"

      - task: Bash@3
        inputs:
            targetType: 'inline'
            script: |
              az pipelines run {PipelineID} --organization $(System.CollectionUri) --project {ProjectName} --parameters apps="[$(Applications.apps)]"
        env:
          AZURE_DEVOPS_EXT_PAT: $(system.accesstoken)

Pipeline Two:

parameters:
- name: apps
  type: object
  default: []

stages:
- stage: ProcessApps
  jobs:
  - job: DummyJob

  - ${{ each app in parameters.apps }}:
    - job: ProcessApp_${{ app.application }}
      displayName: "Processing App '${{ app.application }}'"      
      steps:

      - ${{ each overlay in app.environments }}:
        - task: Bash@3
          displayName: Processing Overlay '${{ overlay }}'
          inputs:            
            targetType: inline
            script: |
              echo ${{ app.serviceconnectionprefix }}-${{ overlay }}

        - task: KubernetesManifest@1
          name: bake_${{ app.application }}_${{ overlay }}
          displayName: Baking ${{ app.application }} - ${{ overlay }}
          inputs:
            action: bake
            renderType: kustomize
            kustomizationPath: '$(System.DefaultWorkingDirectory)/01_applications/${{ app.application }}/overlays/${{ overlay }}'

        - task: KubernetesManifest@1
          displayName: Deploying
          inputs:
            kubernetesServiceConnection: '${{ app.serviceconnectionprefix }}-${{ overlay }}'
            manifests: $(bake_${{ app.application }}_${{ overlay }}.manifestsBundle)