How to Auth to Google Cloud using Service Account in Python?

Question

I'm trying to make a project that will upload Google Storage JSON file to BigQuery (just automate something that is done manually now), and I'd like to use a 'service account' for this as my script is going to be run on daily basis.

After reading everything I can found about using service account, I'm still struggling to authenticate.

Here is what i've done so far:

1. Created JSON key file for service account
2. Installed client libraries: pip install --upgrade google-cloud-bigquery
3. Installed Google Cloud SDK according to: https://cloud.google.com/sdk/docs/
4. Run export GOOGLE_APPLICATION_CREDENTIALS=<path_to_service_account_file> with key path specified correctly

Now I'm trying to run the following python script:

from google.cloud import bigquery
bigquery_client = bigquery.Client()

But I get this error:

google.auth.exceptions.DefaultCredentialsError: Could not automatically determine credentials. Please set GOOGLE_APPLICATION_CREDENTIALS or explicitly create credential and re-run the application. For more information, please see https://developers.google.com/accounts/docs/application-default-credentials.

I'm quite new to both Python and the Google Cloud API so possbily missed something. Can someone point out where/what was wrong in my steps above or point me to clear instruction for dummys about setting up and running simple script with BigQuery using service account?

Answer 1

I usually set this variable in the python script itself, something like:

import os
from google.cloud.bigquery.client import Client

os.environ['GOOGLE_APPLICATION_CREDENTIALS'] = 'path_to_json_file'
bq_client = Client()