google-cloud-platform google-bigquery google-iam

Changing GCP IAM roles for multiple users

I have over one hundred GCP users that have mixed roles. These users were manually created via IAM console by different admin personnel. We would like do some clean up to the project. The roles needed for these users are simply project viewer, bigquery user, bigquery data viewer and bigquery jobuser. I would like to change the roles of these 100+ users with a single command. Is there a way to do this?

Below is an example of IAM list that I retrieved from GCP command

gcloud projects get-iam-policy example-project-id-1

This is simply a dummy data and a modified version for security reasons.

---
policy:
  bindings:
  - members:
    - user:Sanaa.Keller@abc.com
    - user:adam.skill@abc.com
    - user:albert.withmore@abc.com
    - user:alison.mcknight@abc.com
    - user:andre.soretti@abc.com
    - user:andrew.born@abc.com
    - user:andrew.hill@abc.com
    - user:andrew.ives@abc.com
    - user:andrew.windridge@abc.com
    - user:angela.hammond@abc.com
    - user:anil.patel@abc.com
    - user:anna.lynch@abc.com
    - user:ben.carlson@abc.com
    - user:elisabeth.morrison@abc.com

    role: roles/editor
project: projects/926557397521
resource: //cloudresourcemanager.googleapis.com/projects/abc-staging
---
policy:
  bindings:
  - members:
    - user:Sanaa.Keller@abc.com
    - user:adam.skill@abc.com
    - user:albert.withmore@abc.com
    - user:alison.mcknight@abc.com
    - user:andre.soretti@abc.com
    - user:andrew.born@abc.com
    - user:andrew.hill@abc.com
    - user:andrew.ives@abc.com
    - user:andrew.windridge@abc.com
    - user:angela.hammond@abc.com
    - user:anil.patel@abc.com
    - user:anna.lynch@abc.com
    - user:ben.carlson@abc.com
    - user:elisabeth.morrison@abc.com

    roles:roles/bigquery.dataEditor
project: projects/926557397521
resource: //cloudresourcemanager.googleapis.com/projects/abc-staging
---
policy:
  bindings:
  - members:
    - user:Sanaa.Keller@abc.com
    - user:adam.skill@abc.com
    - user:albert.withmore@abc.com
    - user:alison.mcknight@abc.com
    - user:andre.soretti@abc.com
    - user:andrew.born@abc.com
    - user:andrew.hill@abc.com
    - user:andrew.ives@abc.com
    - user:andrew.windridge@abc.com
    - user:angela.hammond@abc.com
    - user:anil.patel@abc.com
    - user:anna.lynch@abc.com
    - user:ben.carlson@abc.com
    - user:elisabeth.morrison@abc.com

    roles:roles/bigquery.jobUser
project: projects/926557397521
resource: //cloudresourcemanager.googleapis.com/projects/abc-staging
---
policy:
  bindings:
  - members:
    - user:brian.king@abc.com
    role: roles/editor
project: projects/926557397521
resource: //iam.googleapis.com/projects/abc-staging/serviceAccounts/qlikview@abc-staging.iam.gserviceaccount.com

So, what i would like to do from the above example is to remove the the project editor role and simply assign them roles of BigQuery Data Viewer and BigQuery Job User

Solution

Since you mentioned in the comment section that this policy binding file contains only the users that you want to change the policies. I was able to create a small process which will help you change your user's policy bindings for your project.

Below it is detailed described with the necessary code to execute each step, in order to change your user's permissions:

1. Get your Policy Bindings file and save it in your shell environment in yaml format, as follows:

gcloud projects get-iam-policy PROJECT_ID --format yaml > policy.yaml

2. Make a copy of the current policy bindings for security reasons with the command

cp policy.yaml your_policy_bindings_file_SECURITY_COPY.yaml

3. Within your Cloud Shell environment create a python file with the below code, name it python_script.py. This script extracts the user's emails from your policy binding YAML file and create a csv file, with the format described here (Upload users from a CSV file). In addition, it creates another .csv file only with the emails. So, it can be used to remove the current policies for each of these users. Below is the code:

import yaml
import re
import pandas as pd
import csv

#declaring the list which will contain all the user's emails from the policy biding yaml
final_list=[]

def yaml_as_python(val):
    """Convert YAML to dict"""
    try:
        return yaml.load_all(val,Loader=yaml.FullLoader)
    except yaml.YAMLError as exc:
        return exc

with open('policy.yaml','r') as input_file, open('emails_to_remove_policies.csv', 'w') as emails_to_remove_policies:
    results = yaml.load_all(input_file,Loader=yaml.FullLoader)

    #print(results)
    for value in results:

        try:
            mydict=value['policy']['bindings'][0]['members']
            mydict=sorted(mydict)
            r=re.compile(r'.+\@(abc.com)')
            my_newlist=[email.lstrip("user:") for email in mydict if re.match(r,email)]
            
            final_list.extend(my_newlist)
            #final_list.writerow(email_list)

            
        except TypeError:
            #print("type erorr")
            continue
            
    #writing the csv with the list of emails which will be removed from the policies        
    wr = csv.writer(emails_to_remove_policies, quoting=csv.QUOTE_NONE, delimiter = '\n')
    wr.writerow(final_list)

#CSV file example from documentation
#Header: Group Email [Required],Member Email,Member Type,Member Role
#Entry: yourgroup@email.com, membername@email.com,USER,MEMBER 


df = pd.DataFrame(final_list)
df2 = pd.DataFrame(final_list, columns=['Member Email'])
df2 = df2.assign(GroupEmail='yourgroup@email.com').assign(MemberType='USER').assign(MemberRole= 'MEMBER')
df2 = df2.reindex(columns=['GroupEmail','Member Email','MemberType','MemberRole'])


df2 = df2.rename(columns={'GroupEmail': 'Group Email [Required]', 'MemberType': 'Member Email', 
                          'MemberRole': 'Member Role' })
#in case you want to check the final output in the terminal
#print (df2)
print("Finished")

#list of emails to add in the google group
df2.to_csv('emails_to_add.csv',index=False)
print("\n\nStarting removing the policies for the list of users\n\n")

Notice that, two .csv files will be created: emails_to_remove_policies.csv and emails_to_add.csv. The first one, will be used within a bash script to remove the current policies for these users from your project. The second will be used to add the emails to a Google Group, then this group will be used to grant the appropriate permissions within your policy bindings project yaml file.

4. Create a file named exec_all.sh with the following code to execute the python_script.py and remove the current policies for the list of unwanted users.

python3 python_script.py

while IFS=, read -r email
do
    for i in $email
        do
        echo "Removing $i"
        gcloud projects remove-iam-policy-binding PROJECT_ID --member user:$i --role ROLE_ID
        #echo "Next"
        done
done < emails_2.csv

From your shell environment execute it by typing: bash exec_all.sh.

5. Now, create a Google Group following the steps described here. Afterwards, add multiple members to your Google Group, you have to provide a .csv with an specific format containing all the users, use the emails_to_add.csv output from step two. Here is the documentation.

In order to download the .csv file to your local environment and then upload it to your Google Group using Google Admin, use the following command:

cloudshell download emails_to_add.csv

6. Get your new Policy Bindings file, after removing the unwanted users in step 4. As follows:

gcloud projects get-iam-policy PROJECT_ID --format yaml > new_policy.yaml

7. Open it, you can use cat, vim or the shell editor, which is the easiest method. Now, in the beginning of the file add:

---
bindings:
- members:
  - user: yourgroup@email.com
  role: role_id
---

Attention to the --- on the first and last line. Also, in order to get the role id, you have to go to: IAM & Admin > Roles (on bottom of the left menu) > Click on the Role you want > You can see the role id next to ID. Finally, notice that the group email is used to grant permission to all the desired users.

8. You have your new Policy Bindings for your project. Thus, you need to set it as the new policy for your project, here is the documentation. You can do it as below:

gcloud projects set-iam-policy PROJECT_ID new_policy.yaml

Notes:

You have to prepare your environment in order to run the codes above, installing the necessary packages. Here is the documentation for it.
Notice that, from your IAM policies from your project, all the unwanted users are removed. Then, they are added back to the project with the group alias and the proper permissions. We do this because, there are other permissions in your project which I assume you do not want to change (also, that is also the reason, we do a security copy in the beginning).