Google Cloud Data Fusion failing to access data from another project, even though the access is granted

Question

I have enabled the Cloud Data Fusion API and created an instance in project A in BigQuery. Project A will contain our datawarehouse and load and transform data from other projects on BigQuery. When I try to create a pipeline with source BigQuery from project B on table B.X, I get the error Unable to get details about the BigQuery table:Access Denied:Table B.X. Permission bigquery.tables.get denied on table B.X (or it may not exist). I have given access to the service account related to the instance of Data Fusion (the red box) in project B as BigQuery Data Viewer, BigQuery Metadata Viewer, BigQuery Read Session User, BigQuery User, Browser, Storage Object Viewer, Viewer

I only need view access on project B to read data and write on project A, so I do not understand why Cloud Data Fusion pipelines would not recognize table X on project B. What could be the issue?

Answer 1

I finally found a solution, I am posting it here for those that might be struggling with the same problem. My service account (the red marked box) was something like cloud-datafusion-management-sa@project- id.iam.gserviceaccount.com which according to this link https://cloud.google.com/data-fusion/docs/concepts/service-accounts#service_account_table, it is deprecated. So how we fixed it, is by checking the project number in project A (let's say it is 1234566) and add service-1234566@gcp-sa- datafusion.iam.gserviceaccount.com to project B as BigQuery Data Viewer, BigQuery Metadata Viewer, BigQuery Read Session User, BigQuery User, Browser, Storage Object Viewer, Viewer. So now I can access the tables of project B.