Search code examples
azureazure-devopsssisazure-pipelinesazure-rm-template

Azure DevOps passing secure string to ARM template


I need to pass a variable of SecureString type in the overRideParameters for a an AzureResourceManagerTemplateDeployment@3 task. The variable is a SAS Token and is created at Container level with read, add, list, write, create permissions.

The ARM template which is auto-generated from Azure Data Factory (ADF) is as follows: enter image description here

If I manually paste the full url to be used in ADF SSIS-IR configuration for Customisation, can successfully start SSIS-IR. However, there is something wrong when the same property values are set via Azure DevOps (ADO) pipeline code where the url and SAS Token are concatenated - SSIS-IR will not start. The error (403) is that the end point cannot be found or do not have permissions.

How can I get this to work?

Image below, showing ADF SSIS configuration with expected SAS token masking.

ADF SSIS configuration with expected SAS token masking

In a task in one yml file I pull the variable (marked as secret) from a Variable Group stored in ADO library. I then map it as required described in Microsoft docs. I then confirm that the value retrieved is as expect:

- job: Deploy_${{parameters.environmentName}}_env
  displayName: Deploy published ADF ARM template 
  dependsOn:   Check_Variables_${{parameters.environmentName}}
  condition:   succeeded()
  variables:
    - group: ado_ssis_cicd_${{parameters.environmentName}} 
    - name:  sassToken_from_group
      value: $(ssisir_sasToken)           
    - template: ../../variables/vars-${{parameters.environmentName}}.yml  
  steps:
  - task: PowerShell@2
    displayName: Extract Library variables.
    inputs:
     targetType: 'inline'
     script: |         
      Write-host "sassToken_from_group from Group_Var from lib: $(sassToken_from_group)" 
      "$(sassToken_from_group)"  -eq "sp=racwl&st=2022-11-07T06:29:55Z&se=2029-11-07T15:29:55Z&spr=https&sv=2021-06-08&sr=c&sig=5pPVjhiNO7g7WQIQb5F2uRFmvU8xJEoao2TSofsQJAA%3D"

The value is passed to another yml as a parameter (type= String), which is then used in the AzureResourceManagerTemplateDeployment@3 task parameter overrides:

- task: AzureResourceManagerTemplateDeployment@3 
  displayName: ARM template deployment - Resource Group scope
  inputs:
    deploymentScope: Resource Group
    azureResourceManagerConnection: ${{ parameters.azureResourceManagerConnection }}
    subscriptionId: ${{ parameters.subscriptionId }}
    resourceGroupName: ${{ parameters.resourceGroupName }}
    location: ${{ parameters.location }}
    csmFile: $(Build.SourcesDirectory)/build/ArmTemplatesDrop/ARMTemplateForFactory.json
    csmParametersFile: $(Build.SourcesDirectory)/build/ArmTemplatesDrop/ARMTemplateParametersForFactory.json
    deploymentMode: Incremental 
    overrideParameters:        
      -ssisir_blobContainerUri ${{ parameters.ssisir_blobContainerUri }} 
      -ssisir_sasToken ${{ parameters.ssisir_token }}

The SAS Token that is pushed to the SSIS customisation configuration is masked as expected, so I have no way of knowing what was deployed.

I know that a secret is just an encrypted string. I do not believe I need (or can) decrypt it. I know that a string that is of SecureString type has memory managed differently to a normal string.

Because the code in the yml file where the parameters are over-ridden is in a step under the same job where the variable is pulled & mapped, I have also tried using the variable directly rather than the value passed in via parameter:

  overrideParameters:  
      -ssisir_blobContainerUri ${{ parameters.ssisir_blobContainerUri }} 
      -ssisir_sasToken $(sassToken_from_group)

Solution

  • Problem resolved. In my opinion this is a bug. The url to the blob container needs a '?' appended such that when it is generated and pushed into ADF the blob container url and the SAS token parameter are separated as expected in standard URI format.

    Below is what is generated in the ARM templates when publish from ADF. Note that there is no '?' character appended. One would think that the code would appends the character when it generates what is deployed to the target field.

    enter image description here