Best way to do a batch removal of a list of resouce groups

Question

I'm currently automating some clean-up operations in Azure and the structure of my code is basically:

• A huge for loop that iterates over a list of resource groups that should be deleted.
• If any of those groups is missing a list of required tags defined by me, delete that resource group.

Currently the deletion operation is as simple as:

// Inside the for loop that iterates over a list of resource group names

elif rg.tags.get('delete_at', False):

    if datetime.now() > datetime.strptime(rg.tags['delete_at'], '%Y-%m-%d %H:%M:%S'):
        print('Resource Group: ' + rg.name + ' is going to be deleted')
        try:
            delete_async_op = delete_rg(rg_client, rg.name)
            delete_async_op.wait()
        except CloudError:
            # If any of the delete operation fails, don't block the policy execution over the rest of resource groups
            print("The delete operation for the resource group: '" + rg.name + "' didn't suceed. Details: " + traceback.format_exc())
            continue

Just for the recor,d the delete_rg function its just a wrapper that performs the following operation:

rg_client.resource_groups.delete(rg_name)

So I was wondering if there would be a more efficient way to do this. Maybe having the list of names of the resource groups that should be deleted and creating threads for those operations? Just guessing, looking for some advice.

Many thanks in advance! Let me know if my question is not clear enough.

Answer 1

The delete operation is actually creating a thread already for you, and you join on the thread when you do "wait()". I would suggest you keep delete_async_op in a list, and you start them all, then you wait.

Something like:

delete_list = []

# your main loop

elif rg.tags.get('delete_at', False):

    if datetime.now() > datetime.strptime(rg.tags['delete_at'], '%Y-%m-%d %H:%M:%S'):
        print('Resource Group: ' + rg.name + ' is going to be deleted')
        try:
            delete_list.append(delete_rg(rg_client, rg.name))
        except CloudError:
            # If any of the delete operation fails, don't block the policy execution over the rest of resource groups
            print("The delete operation for the resource group: '" + rg.name + "' didn't suceed. Details: " + traceback.format_exc())
            continue

# out of main loop

for op in delete_list:
    try:
        op.wait()
    except CloudError:
        print("Unable to confirm deletion")

The initial call to ask for deletion is done in your "if" (it's why you want to catch CloudError). The polling using thread is done in wait(), but since there is actual call to Azure, you still need to catch CloudError in case anything happens.

(I work at MS in the Python SDK team)