Search code examples
pythongoogle-apigoogle-bigquerygoogle-oauthgoogle-api-python-client

Login Required error using service account Google BigQuery API


I wrote a python script that called insert API of BigQuery to insert data to a table. I use service account for OAuth authentication. It functioned correctly for few days but now it gives Login Required error (401). Following is the code:

scopes = ['https://www.googleapis.com/auth/bigquery']

credentials =ServiceAccountCredentials.from_json_keyfile_name('/home/ubuntu/aha/udofy.json', scopes=scopes)

service = discovery.build('bigquery', 'v2', credentials=credentials)

job_body = { } #Body JSON

request = service.jobs().insert(projectId=projectId, body=job_body)

response = request.execute()

I see the following error when I check the job logs on the UI:

{

"error": {

 "errors": [
   {

    "domain": "global",
    "reason": "required",
    "message": "Login Required",
    "locationType": "header",
    "location": "Authorization"
   }
  ],

  "code": 401,

  "message": "Login Required"

 }

}

API Response:

{u'configuration': {u'query': {u'createDisposition': u'CREATE_IF_NEEDED',
                               u'destinationTable': {u'datasetId': u'co_gradeup_android_ANDROID',
                                                     u'projectId': u'udofy-1021',
                                                     u'tableId': u'user_answer_attempts'},
                               u'query': u' SELECT user_dim.user_id as user_id, event_dim.params.value.string_value as post_id, DATE(USEC_TO_TIMESTAMP(event_dim.timestamp_micros)) as attempt_date, count(*) as attempt_count FROM [udofy-1021:co_gradeup_android_ANDROID.app_events_20161101] WHERE event_dim.name =\'Answer_Selected\' and event_dim.params.key = \'postId\' and user_dim.user_id is not null and user_dim.user_id != "" group by 1,2,3 ',
                               u'writeDisposition': u'WRITE_APPEND'}},
 u'etag': u'', #User_etag
 u'id': u'', #User_Id
 u'jobReference': {u'jobId': u'job_C9kSTaEOQBw0VPBEF_Yj44C0-Us',
                   u'projectId': u'udofy-1021'},
 u'kind': u'bigquery#job',
 u'selfLink': u'https://www.googleapis.com/bigquery/v2/projects/udofy-1021/jobs/job_C9kSTaEOQBw0VPBEF_Yj44C0-Us',
 u'statistics': {u'creationTime': u'1478082742163',
                 u'startTime': u'1478082742420'},
 u'status': {u'state': u'RUNNING'},
 u'user_email': u''} #service_account_email

Solution

  • The issue was that the output schema/format of the query in the job body did not match the schema of the table in which the data was being inserted via the insert API. There was a minor change in the query which was executed in the job body.

    The output format of the query should be exact same as the table schema in the the output data has to be inserted.