Parsing an output variable, across YAML pipeline, from a powershell script, into a Terraform variable

Question

I have been working on a solution to send Azure alerts to Slack, using a logic app (in order to transform the output from the alert into a JSON schema that Slack can display as a message.) The logic app has been deployed as an ARM template, in order to preserve the contents of the logic app fully, while the rest of the Azure resources are being deployed by Terraform. The Terraform and ARM template is being deploying with a Azure DevOps YAML pipeline, with multiple stages. So far I have written the logic app to transform alerts into messages (the logic app posts the message when the schema has been transformed.)

My current dilemma is how to programmatically include the URL of the logic app (where the alerts should be sent) in the Terraform configuration. This is made harder by the fact that there is no config attribute for the URL in the available data blocks for logic app workflows, or for a standard logic app instance.

In order to mitigate this lack of Terraform functionality, I have attempted to retrieve the logic app URL with the Az Powershell module command (the Azure CLI doesn't appear to have the functionality yet.) Using a short script I am able to get the url to trigger the logic app:

$logicApp = Get-AzLogicAppTriggerCallbackUrl -ResourceGroupName "logic-app-rg" -Name "mylogicapp" -TriggerName "Manual"
$url = logicApp.Value

By adding the following line, this can be added into the YAML pipeline:

write-host "##vso[task.setvariable variable=outputURL;isOutput=true]$url"

As there are multiple stages, and only 1 logic app needed, it is placed in the first stage, where core infrastructure is created (storage account for Terraform state.)

The difficulty arises when I am unable to send the data from the task that outputs the URL to a different stage which contains the terraform. The rough structure of the YAML pipeline (simplified):

stages:
- stage: infra-1
  jobs:
  - job: deploy-common-infra
    steps:
    - script: |
        cd core-infra
        terraform init
        terraform plan
        terraform apply
        $logicApp = Get-AzLogicAppTriggerCallbackUrl -ResourceGroupName "logic-app-rg" -Name "mylogicapp" -TriggerName "Manual"
        $url = logicApp.Value
        write-host "##vso[task.setvariable variable=outputURL;isOutput=true]$url"
      name: getLogicAppURL
- stage: build
  jobs:
  - job: build
    - task: build-app
- stage: infra-2
  dependsOn:
  - infrastructure-1
  variables:
    outputURL: $[stageDependencies.infra-1.deploy-common-infra.outputs['getLogicAppURL.outputURL']]
  jobs:
  - job: deploy-infra
    - script: |
        cd infra
        terraform init
        terraform plan -var="logicAppUrl='$(outputURL)'"
        terraform apply

It should be noted that in the real pipeline, I am using dedicated Terraform tasks, as opposed to writing Terraform commands in scripts.

The main part of my trouble comes as I don't want to skip the "build" stage, otherwise I won't have an app being deployed at the last stage (excluded from the above example pipeline.) In addition, the value that is sent to terraform is "null" (there is no URL sent!)

I have used and looked at existing answers for how to share variables across jobs and stages, and while using dependencies in a pipeline, but have so far struggled to find a solution that allows me to parse the URL variable across stages. Is this the only way to parse a YAML variable across stages, into Terraform?

(An additional question might be is this the best approach to the challenge at hand, or is there a different solution that I should be attempting?)

Answer 1

I ended up making a work-around solution to this problem by adding the script to the specific job, with a condition that will only run for that specific Terraform module, and changing the script so it would write to the terraform file directly. For example, to build on the pipeline above with the solution I now have at hand:

stages:
- stage: infra-1
  jobs:
  - job: deploy-common-infra
    steps:
    - script: |
        cd core-infra
        terraform init
        terraform plan
        terraform apply
- stage: build
  jobs:
  - job: build
    steps:
    - task: build-app
- stage: infra-2
  dependsOn:
  - infrastructure-1
  variables:
    outputURL: $[stageDependencies.infra-1.deploy-common-infra.outputs['getLogicAppURL.outputURL']]
  jobs:
  - job: deploy-infra
    steps:
    - template: logic-app.yaml
    - script: |
        cd infra
        terraform init
        terraform plan -var="logicAppUrl='$(outputURL)'"
        terraform apply

The logic-app.yaml template has the script to add the url to the variable file, which is stored in a temporary storage location on the Agent used by DevOps:

steps:
- script: |
    $logicApp = Get-AzLogicAppTriggerCallbackUrl -ResourceGroupName "logic-app-rg" -Name "mylogicapp" -TriggerName "Manual"
    $url = $logicApp.Value
    Add-Content -Path "$directory/infra/vars.tfvars" -Value "`nslack_url=`"$url`""

It should be noted that terraform/vars.tfvars is the path in the repository that was being used for the pipeline. By adding this to the pipeline, it now runs successfully.