I have several customer projects that write analytic events into a BigQuery dataset. The setup is organised like this:
1) Each GCP project has its own set of GCP resources and some of them report analytics using BigQuery insert API.
2) There's a single "Main Analytics" project that intakes all the data from the different projects in a standardised table (all projects write in the same data format).
I've created a custom IAM role in "Main Analytics" with the required permissions to execute a row insert operation:
bigquery.datasets.get
bigquery.tables.get
bigquery.tables.updateData
For every customer project I've created a unique service account with the above role. This allows each resource in any project to authenticate and insert rows (but not create/delete tables).
Problem: What I really want to do is limit the service accounts to write only to a specific dataset that intakes all the data. The above IAM role allows the service account to list all datasets/tables in the "Main Analytics" project and to insert into them.
If I use dataset permissions - add the service account email as a user to the dataset ACL - then it would have to be WRITER dataset role which would allow the service account to create & delete tables in the dataset which is too broad.
Combining the IAM role with the dataset permissions results in a union so the wider WRITER permission take effect over the narrower IAM role.
Anyway I can configure roles/permissions to allow each service account to insert and only-insert to a specific dataset?
You can drop the bigquery.datasets.get permission from the custom IAM role so that they can’t list all the datasets, and then in the dataset's permissions give the READER role instead of WRITER to the user for that specific dataset.