Search code examples
google-cloud-platformgoogle-bigquerygoogle-cloud-data-fusion

Connect BigQuery as a source to Data Fusion in another GCP project

I am trying to connect BigQuery of ProjectA to Data Fusion of ProjectB and its asking me to enter a service key file. I have tried to upload the service key file to Cloud Storage of ProjectB and provided the link but it's asking me to provide a local file path.

Can someone help me on this?

Thanks in advance.

enter image description here

Solution

  • Can you try this, grant BQ permission of project A to data fusion in project B.

    1. service-project_number@gcp-sa-datafusion.iam.gserviceaccount.com.
    2. project_number-compute@developer.gserviceaccount.com.

    Steps:

    1. Navigate to the customer project that contains the CDF instance and copy the project number (this is found on the Home Page in the Project Info card)

    2. Navigate to the project that contains the resources you would like to interact with.

    3. In the sidebar, click on ‘IAM & Admin’

    4. Click on ‘Add’ at the top of the page.

    5. Provide the first service account name from the table above, be sure to replace with the actual number you obtained in step 1

    6. Grant the Admin role for the resource you would like to interact with. Ex. BigQuery Admin for reading/writing to BigQuery. For BigQuery, you will also need to grant the BigQuery Data Owner role as well.

    7. Repeat steps 5 & 6 for the second service account in the table above.

    8. In your pipeline, ensure you define the correct Project Id for the sources/sinks. Using ‘auto-detect’ will default to the customer project that contains the CDF instance.