I need to pass a variable of SecureString type in the overRideParameters for a an AzureResourceManagerTemplateDeployment@3 task. The variable is a SAS Token and is created at Container level with read, add, list, write, create permissions.
The ARM template which is auto-generated from Azure Data Factory (ADF) is as follows:
If I manually paste the full url to be used in ADF SSIS-IR configuration for Customisation, can successfully start SSIS-IR. However, there is something wrong when the same property values are set via Azure DevOps (ADO) pipeline code where the url and SAS Token are concatenated - SSIS-IR will not start. The error (403) is that the end point cannot be found or do not have permissions.
How can I get this to work?
Image below, showing ADF SSIS configuration with expected SAS token masking.
In a task in one yml file I pull the variable (marked as secret) from a Variable Group stored in ADO library. I then map it as required described in Microsoft docs. I then confirm that the value retrieved is as expect:
- job: Deploy_${{parameters.environmentName}}_env
displayName: Deploy published ADF ARM template
dependsOn: Check_Variables_${{parameters.environmentName}}
condition: succeeded()
variables:
- group: ado_ssis_cicd_${{parameters.environmentName}}
- name: sassToken_from_group
value: $(ssisir_sasToken)
- template: ../../variables/vars-${{parameters.environmentName}}.yml
steps:
- task: PowerShell@2
displayName: Extract Library variables.
inputs:
targetType: 'inline'
script: |
Write-host "sassToken_from_group from Group_Var from lib: $(sassToken_from_group)"
"$(sassToken_from_group)" -eq "sp=racwl&st=2022-11-07T06:29:55Z&se=2029-11-07T15:29:55Z&spr=https&sv=2021-06-08&sr=c&sig=5pPVjhiNO7g7WQIQb5F2uRFmvU8xJEoao2TSofsQJAA%3D"
The value is passed to another yml as a parameter (type= String), which is then used in the AzureResourceManagerTemplateDeployment@3 task parameter overrides:
- task: AzureResourceManagerTemplateDeployment@3
displayName: ARM template deployment - Resource Group scope
inputs:
deploymentScope: Resource Group
azureResourceManagerConnection: ${{ parameters.azureResourceManagerConnection }}
subscriptionId: ${{ parameters.subscriptionId }}
resourceGroupName: ${{ parameters.resourceGroupName }}
location: ${{ parameters.location }}
csmFile: $(Build.SourcesDirectory)/build/ArmTemplatesDrop/ARMTemplateForFactory.json
csmParametersFile: $(Build.SourcesDirectory)/build/ArmTemplatesDrop/ARMTemplateParametersForFactory.json
deploymentMode: Incremental
overrideParameters:
-ssisir_blobContainerUri ${{ parameters.ssisir_blobContainerUri }}
-ssisir_sasToken ${{ parameters.ssisir_token }}
The SAS Token that is pushed to the SSIS customisation configuration is masked as expected, so I have no way of knowing what was deployed.
I know that a secret is just an encrypted string. I do not believe I need (or can) decrypt it. I know that a string that is of SecureString type has memory managed differently to a normal string.
Because the code in the yml file where the parameters are over-ridden is in a step under the same job where the variable is pulled & mapped, I have also tried using the variable directly rather than the value passed in via parameter:
overrideParameters:
-ssisir_blobContainerUri ${{ parameters.ssisir_blobContainerUri }}
-ssisir_sasToken $(sassToken_from_group)
Problem resolved. In my opinion this is a bug. The url to the blob container needs a '?' appended such that when it is generated and pushed into ADF the blob container url and the SAS token parameter are separated as expected in standard URI format.
Below is what is generated in the ARM templates when publish from ADF. Note that there is no '?' character appended. One would think that the code would appends the character when it generates what is deployed to the target field.