Azure resource tag bulk update using Azuredevops pipeline

Question

I have created a pipeline by following the answer here and my requirement is to override all the current tags of each resources with my inputs in the pipeline. So I decided to use the az command with "replace" operation so that the tags will be always replaced with whatever input values we are given, and which will help me to always confirm from this pipeline what tags are applied to each resources.

But as given in the below yaml, the "foreach" is loop is not working for me the way which I expected. Below are the concerns

trigger:
- none

pool:
  name: mypool
parameters:
- name: myEnvironments
  type: object
  default:
  - development
- name: tagList
  type: object
  default:
  - resourcename:  resource1
    todeploy: yes
    allure_envtest_1: allure_envtest_1_value
    allure_envtest_2: allure envtest value

  - resourcename: resource 2
    todeploy: yes
    trip_envtest_1: trip_envtest_1_value
    trip_envtest_2: trip_envtest_2_value

stages:
- ${{ each environment in parameters.myEnvironments }}:
  - stage: 
    displayName: 'Create Tag in ${{ environment }}'
    pool:
      name: mypool
    jobs:
      - ${{ each tag in parameters.tagList }}:
        - ${{ each tagcontent in tag }}:
          - ${{ if and(ne(tagcontent.Key, 'resourcename'),ne(tagcontent.Key, 'todeploy')) }}:
            - job:
              displayName: 'Tag the reource ${{ tag.resourcename }}'
              steps:
              - task: AzureCLI@2
                displayName: "Tag the resource"
                inputs:
                  azureSubscription: ""
                  scriptType: 'bash'
                  scriptLocation: 'inlineScript'
                  inlineScript: 'az tag update --resource-id ${{ tag.resourcename }} --operation replace --tags ${{ tagcontent.Key }}=${{ tagcontent.value }}'
            

  
              

• Since I used "Replace" operation in az command, its always replacing with the last value from tagList, because all the previous values are replaced with this last value.

• When I am adding some spaces to the tag values, which is creating unexpected tags. how to use spaces in these tag values

• the above for each tasks are creating multiple tasks for each tags and I will have many resources as input with more than 10 tags each, So which will result very big pipeline and difficult to manage. So whether we can consolidate the steps in each task in better way.

• Any ways to verify or validate the results of tagging , before its getting applied.

Answer 1

Since I used "Replace" operation in az command, its always replacing with the last value from tagList, because all the previous values are replaced with this last value.

When I am adding some spaces to the tag values, which is creating unexpected tags. how to use spaces in these tag values

These situation comes from the usage you are using is not correct.

From your description, looks like you want add the tags, I create two resource groups to test, write a YAML file and it works fine.

trigger:
- none

pool:
  vmImage: ubuntu-latest
variables:
 YAML_Path: azure-pipelines.yml
parameters:
- name: myEnvironments
  type: object
  default:
  - development
- name: tagList
  type: object
  default:
  - resourcename:  /subscriptions/xxx/resourcegroups/testgroup1
    todeploy: yes
    allure_envtest_1: allure_envtest_1_value
    allure_envtest_2: allure envtest value

  - resourcename: /subscriptions/xxx/resourcegroups/testgroup2
    todeploy: yes
    trip_envtest_1: trip_envtest_1_value
    trip_envtest_2: trip_envtest_2_value
                    
stages:
- ${{ each environment in parameters.myEnvironments }}:
  - stage: 
    displayName: 'Create Tag in ${{ environment }}'
    pool:
      vmImage: ubuntu-latest
    jobs:
      - ${{ each tag in parameters.tagList }}:
        - job: 
          displayName: 'Remove tags from resource' #This will achieve the same feature as replace operation, remove tags first.
          steps:
          - task: AzureCLI@2
            displayName: "Remove tags from the resource"
            inputs:
              azureSubscription: 'xxx'
              scriptType: 'bash'
              scriptLocation: 'inlineScript'
              inlineScript: | # Add "" in this place and use merge operation.
                  az tag delete --resource-id ${{ tag.resourcename }} --yes -y
        - ${{ each tagcontent in tag }}:
          - ${{ if and(ne(tagcontent.Key, 'resourcename'),ne(tagcontent.Key, 'todeploy')) }}:
            - job:
              displayName: 'Tag the reource ${{ tag.resourcename }}'
              steps:
              - task: AzureCLI@2
                displayName: "Tag the resource"
                inputs:
                  azureSubscription: 'testbowman_in_AAD'
                  scriptType: 'bash'
                  scriptLocation: 'inlineScript'
                  inlineScript: | # Add "" in this place and use merge operation.
                      az tag update --resource-id ${{ tag.resourcename }} --operation merge --tags "${{ tagcontent.Key }}"="${{ tagcontent.value }}"

The above YAML will be able to achieve the 'add tags' requirement(It can remove the tags before adding them, just like the replace operation.):

the above for each tasks are creating multiple tasks for each tags and I will have many resources as input with more than 10 tags each, So which will result very big pipeline and difficult to manage. So whether we can consolidate the steps in each task in better way.

Just to emphasize that, the usage of the loop you are using in DevOps YAML pipeline is different from loop in Usual develop language. It was named Conditional Insertion, this feature can't pop out a value or increase on the same value.

The YAML I shared at the beginning can already achieve the functions you need. In your case, if there are a lot of tags, a feasible way is to use scripts in the pipeline to parse the YAML content and get all the key-value pairs related to tags. After that, you can generate a string variable based on all of the key-value pairs. And through the logging command, it will be exposed to other tasks under the concept of YAML pipeline run. You can use the exposed string variable in your Azure CLI Task.

I write a Python script for you to get the tags in parameters:

import yaml

def get_yaml_tagList_parameters(yaml_file):
    with open(yaml_file, 'r') as stream:
        try:
            yaml_parameters = yaml.load(stream, Loader=yaml.FullLoader)
            #get parameters from yaml file
            tagList = yaml_parameters['parameters'][1]['default']
            return tagList
        except yaml.YAMLError as exc:
            print(exc)

#define the YAML file path
yaml_file = "YAML_Files/xxx.yml"
tagList = get_yaml_tagList_parameters(yaml_file)
#get the length of the list
list_length = len(tagList)
#create a string list with the length of the list
string_list = [str] * list_length
# print(list_length)
for tag in tagList:
    for key, value in tag.items():
        if key != 'resourcename' and key != 'todeploy':
            #get the array index of the tagList
            index = tagList.index(tag)
            #increase the key and value to the string list based on the index
            
            string_list[index] = str(string_list[index]) + key + "=" + "\"" + value + "\""+" "

for string in string_list:
    #remove the space at the end of the string
    print(string.replace("<class 'str'>", ""))

The result should be like this(Make sure the value part is surrounded by double quotes):

The design of this will be difficult, but it's inevitable because Condition Insertion is not so flexible.

You can design your own code, above is just a sample.

Any ways to verify or validate the results of tagging , before its getting applied.

The answer is NO.