I'm new to Azure Pipelines and I'm trying to handle the same variable name in different variable groups. I can't be too specific for security reasons, but the general gist is that I have multiple environments that I access via APIs, and I would like to be able transfer data any which way between these environments. The general set up is as such:
I have variable groups of different names (there are currently 2 for this poc, but will be 10-12 in prod)
'alpha'
'beta'
that contain the same variables:
'env' # hoping to use as a key from parameters to fetch this data
'url' # api url
'token' # api key
I would like to be able to get a source and destination environment from the user, and then set variables within the pipe sourceUrl, sourceEnv and sourceToken - destUrl, destEnv, destToken from these variable groups.
I'm aware on the documentation the variables with the same name are just set to the value of the last listed variable group, so I'm wondering if I have to use a bash script to set these variables dynamically. Hopefully the yaml makes this a little clearer:
trigger: none
parameters:
- name: sourceEnv
type: string
default: alphaDev
values:
- alphaDev
- betaDev
displayName: 'Source Environment'
- name: destEnv
type: string
default: betaDev
values:
- alphaDev
- betaDev
displayName: 'Destination Environment'
variables:
- group: ${{ parameters.sourceEnv }} # i want to set sourceUrl sourceEnv and sourceToken to these values
- group: ${{ parameters.destEnv }} # i want to set desturl destEnv and destToken to these values
# so essentially the variables would be:
- name: sourceUrl
value: ${{ parameters.sourceEnv}}.$(url)
- name: sourceToken
value: ${{ parameters.sourceEnv}}.$(token)
# and same for destination
I have seen that you can directly declare the variable groups inside of jobs, which is what I'm doing currently with the POC.
However as the same variables are required across multiple jobs I wondered if there's a way to declare them once at the start of the file. Granted, this is mainly for maintainability, as once this is completed it will be others who manage the code.
As for the code, I wondered if setting variables immediately after calling the group might work
variables:
- group: ${{ parameters.sourceEnv }}
- name: sourceUrl
value: $(url)
- group: ${{ parameters.destEnv }}
- name: destUrl
value: $(url)
but this just sets them both to the destEnv url value.
Hopefully my requirements are clear, and perhaps this question is incredibly novel, but as I said I'm new to azure pipelines and am hoping someone is able to assist with with say a bash script to make this possible, unless there's a straight forward way I've missed :)
Based on your description, you have multiple variables with the same name in different variable groups.
Refer to this doc: Manage variable groups
If multiple variable groups include the same variable, the last variable group that uses the variable in the file sets the variable's value.
In this case, if you specify two variable groups directly in the root YAML node(based on your example in the question), it will use the value in the destEnv variable group. This is an expected behavior.
However as the same variables are required across multiple jobs I wondered if there's a way to declare them once at the start of the file.
For the reasons stated above, I'm afraid there is not such method can directly achieve this requirement.
so I'm wondering if I have to use a bash script to set these variables dynamically.
Yes. This is required. You need to set the variables group in two different jobs/stages and use cross jobs/stages variables to map the correct value to source and dest env varaibles. Refer to this doc: Use output variables from tasks
Here is an example:
parameters:
- name: sourceEnv
type: string
default: alphaDev
values:
- alphaDev
- betaDev
displayName: 'Source Environment'
- name: destEnv
type: string
default: betaDev
values:
- alphaDev
- betaDev
displayName: 'Destination Environment'
jobs:
- job: SourceEnv
dependsOn:
variables:
- group: ${{ parameters.sourceEnv }}
steps:
- bash: |
echo "##vso[task.setvariable variable=sourceUrl;isOutput=true]${{ parameters.sourceEnv}}.$(url)"
echo "##vso[task.setvariable variable=sourceToken;isOutput=true]${{ parameters.sourceEnv}}.$(token)"
name: setvarStep
- job: DestEnv
dependsOn:
variables:
- group: ${{ parameters.destEnv }}
steps:
- bash: |
echo "##vso[task.setvariable variable=destUrl;isOutput=true]${{ parameters.destEnv}}.$(url)"
echo "##vso[task.setvariable variable=destToken;isOutput=true]${{ parameters.destEnv}}.$(token)"
name: setvarStep
- job: UseVariables
dependsOn:
- SourceEnv
- DestEnv
variables:
sourceUrl: "$[ dependencies.SourceEnv.outputs['setvarStep.sourceUrl'] ]"
sourceToken: "$[ dependencies.SourceEnv.outputs['setvarStep.sourceToken'] ]"
destUrl: "$[ dependencies.DestEnv.outputs['setvarStep.destUrl'] ]"
destToken: "$[ dependencies.DestEnv.outputs['setvarStep.destToken'] ]"
steps:
- script: |
echo $(sourceUrl)
echo $(sourceToken)
echo $(destUrl)
echo $(destToken)
On the other hand, whether we use classic mode or YAML, it is not recommended to define a variable with the same name in different variable groups. You can also consider setting the variable name based on the environment name in variable group.