Search code examples
google-bigquerygsutilpybigquery

BigQuery Client Url

My server is a closed-network and i want to make a whitelist only for bigquery client and gsutil.

I have tried adding following url

    https://cloud.google.com/bigquery
    https://console.cloud.google.com/

But it turns out bigquery needs a little more url for authenticating purposes and many more. For example like the urls below

https://accounts.google.com/o/oauth2/auth
https://www.googleapis.com/oauth2/v1/certs

Does anyone know all the url that is used for bigquery-client and gsutil. Not only for authenticating but also for creating jobs and fetching data from storage

Solution

  • Per the official documentation you can see that the URL https://www.googleapis.com/auth/bigquery is the one used for BigQuery to authenticate:

    #This is a Python example
appflow = flow.InstalledAppFlow.from_client_secrets_file(
    "client_secrets.json", scopes=["https://www.googleapis.com/auth/bigquery"]
)

    But, then you need to whitelist the URL for the API authorization of your project, for example:

    curl -H "Authorization: Bearer $ACCESS_TOKEN" \
 "https://www.googleapis.com/bigquery/v2/projects/$GOOGLE_CLOUD_PROJECT/datasets"

    I recommend you to whitelist all of GCS/Google APIs or if you want to be more detailed deploy a firewall and see the traffic you are getting from GCS/BigQuery