Writing a secret from AzureKeyVault in a Azure DevOps Pipeline to a file with powershell

I retrieve a secret from the Azure KeyVault in my Azure DevOps pipeline and need to write it as plain text to a file (~/.databrickscfg) in order to authenticate against a (databricks) service.

Currently I get the following error Error: failed during request visitor: inner token: Post "https://xxx-xxxxxxxxxxxxxxxxxx.x.azuredatabricks.net/oidc/v1/token": {"error":"invalid_client","error_id":"xxxxx-xxxx-xxxx-xxxx-xxxxxx","error_description":"Client authentication failed"}

I initially though that it actually writes the content of the secret variable as *** to the file, while I need it as plain text. I tried to convert it to a normal string with ConvertFrom-SecureString:

echo $(ConvertFrom-SecureString -SecureString "$(mysecret-id-${{ parameters.env }})" -AsPlainText)

but it failed with this error Cannot bind parameter 'SecureString'. Cannot convert the value of type "System.String" to type "System.Security.SecureString". So it seems, like it's already a normal String and only the logger obfusciates the output.

However, at the moment, I get an authentication error due to the secret not being correct when I try to insert it into the file from the KeyVault variable.

If I hardcode the id and secret in the YAML pipeline, it works. So it's an issue of how can I write the secret as plain text to a file?

Here is my pipeline:

parameters:
- name: env
  displayName: Select environment
  type: string
  default: P
  values:
  - DT
  - P

steps:
- task: AzureKeyVault@2
  displayName: 'Get secret: DevOps SP dbx secret 1'
  inputs:
    azureSubscription: 'my-subcription'
    keyVaultName: 'my-keyvault-name'
    secretsFilter: 'mysecret-${{ parameters.env }}'
    runAsPreJob: false
- task: AzureKeyVault@2
  displayName: 'Get secret: DevOps SP dbx secret 2'
  inputs:
    azureSubscription: 'my-subcription'
    keyVaultName: 'my-keyvault-name'
    secretsFilter: 'mysecret-id-${{ parameters.env }}'
    runAsPreJob: false


- powershell: |

    $databricks_host_line = "host = https://xxx-xxxxxxxxxx.x.azuredatabricks.net"
    $dbx_cli_profile = "profilename-${{ lower(parameters.env) }}"
    $client_secret_line = "client_secret = " + "$(mysecret-${{ parameters.env }})"
    $client_id_line = "client_id = " + "$(mysecret-id-${{ parameters.env }})"
    # $client_secret_line = "abc"  # if I write it manually in plain text it works.
    # $client_id_line = "abc" # if I write it manually in plain text it works.

    touch ~/.databrickscfg
    echo "[$dbx_cli_profile]" >> ~/.databrickscfg
    echo $databricks_host_line >> ~/.databrickscfg
    echo $client_id_line >> ~/.databrickscfg
    echo $client_secret_line >> ~/.databrickscfg
    echo "" >> ~/.databrickscfg

Solution

You can map the secrets to environment variables in your PowerShell task. For example,

- powershell: |
      $databricks_host_line = "host = https://xxx-xxxxxxxxxx.x.azuredatabricks.net"
      $dbx_cli_profile = "profilename-${{ lower(parameters.env) }}"
      $client_secret_line = "client_secret = " + "$env:mysecret"
      $client_id_line = "client_id = " + "$env:myclient"
      
      touch ~/.databrickscfg
      echo "[$dbx_cli_profile]" >> ~/.databrickscfg
      echo $databricks_host_line >> ~/.databrickscfg
      echo $client_id_line >> ~/.databrickscfg
      echo $client_secret_line >> ~/.databrickscfg
      echo "" >> ~/.databrickscfg
  env:
    mysecret: $(mysecret-${{ parameters.env }})
    myclient: $(mysecret-id-${{ parameters.env }})