Cannot set ENV variable in azure pipeline with value from other step

Question

So I am generating an access token from Databricks to test connection in CI/CD azure devops.

I am able to generate the token, i am able to echo it in another step (for testing purposes). But one step uses env: to list environment variables and I need to pass this token to the environment variable but that does not seem to work.

Here is that part of the pipeline.

- task: AzurePowerShell@5
  displayName: 'Generate Databricks credentials'
  inputs:
    azureSubscription: ${{ parameters.azureServiceConnection }}
    azurePowerShellVersion: 'LatestVersion'
    ScriptType: 'InlineScript'
    Inline: |
      $token = (Get-AzAccessToken -Resource "2ff814a6-3304-4ab8-85cb-cd0e6f879c1d").Token
      Write-Host "##vso\[task.setVariable variable=DATABRICKS_TOKEN;isOutput=true;isSecret=false\]$token"
  name: GenerateTokenStep

- script: echo $(GenerateTokenStep.DATABRICKS_TOKEN) \<--- this works
  displayName: 'Echo Databricks token'

- task: AzureCLI@2
  displayName: "dbt debug"
  inputs:
    azureSubscription: ${{ parameters.azureServiceConnection }}
    ScriptType: bash
    scriptLocation: inlineScript
    inlineScript: |
      echo "dbx token: $(GenerateTokenStep.DATABRICKS_TOKEN)"
      dbt debug --profiles-dir $(location)
  env:
    AZURE_SUBSCRIPTION: $(AZURE_SUBSCRIPTION)
    DATABRICKS_HOSTNAME: $(DATABRICKS_HOSTNAME)
    DATABRICKS_HTTP_PATH: $(DATABRICKS_HTTP_PATH)
    DATABRICKS_SCHEMA: $(DATABRICKS_SCHEMA)
    DATABRICKS_ACCESS_TOKEN: $(GenerateTokenStep.DATABRICKS_TOKEN) \<--- this doesnt seem to work

Anyone has an idea why this is not working?

I tried changing the variable set line to this:

echo "##vso[task.setVariable variable=DATABRICKS_TOKEN;isOutput=true;isSecret=false]$token"

But this seems to have the same behaviour.

Answer 1

You are doing in correct way, the yaml works on my side.

Only removed extra \ when setting variables in AzurePowerShell@5 task based on your yaml:

pool:
  vmImage: Ubuntu-latest

steps:
- task: AzurePowerShell@5
  displayName: 'Generate Databricks credentials'
  inputs:
    azureSubscription: 'ARMConn4'
    azurePowerShellVersion: 'LatestVersion'
    ScriptType: 'InlineScript'
    Inline: |
      $token = (Get-AzAccessToken -Resource "2ff814a6-3304-4ab8-85cb-cd0e6f879c1d").Token
      Write-Host "##vso[task.setvariable variable=DATABRICKS_TOKEN;isOutput=true;isSecret=false]$token"
  name: GenerateTokenStep

- script: echo $(GenerateTokenStep.DATABRICKS_TOKEN) #<--- this works
  displayName: 'Echo Databricks token'

- task: AzureCLI@2
  displayName: "dbt debug"
  inputs:
    azureSubscription: 'ARMConn4'
    scriptType: 'bash'
    scriptLocation: 'inlineScript'
    inlineScript: |
      echo "dbx token: $(GenerateTokenStep.DATABRICKS_TOKEN)"
      echo "dbx token: $DATABRICKS_ACCESS_TOKEN"
  env:
    DATABRICKS_ACCESS_TOKEN: $(GenerateTokenStep.DATABRICKS_TOKEN) 

To use the variable, you can use $(GenerateTokenStep.DATABRICKS_TOKEN), or read from env as $DATABRICKS_ACCESS_TOKEN as you set in last AzureCLI@2 task. Please check the doc Access variables through the environment.

It's recommended to copy my yaml for a test. I used Microsoft-hosted agent to avoid any local agent influence.