Search code examples
google-cloud-platformgoogle-bigquerygoogle-oauthvelo

Streaming data to BigQuery from Corvid (node.js npm) - Request is missing required authentication credential ERROR


I'm trying to stream data to my BigQuery table using Corvid

So I installed BigQuery model to node. enter image description here

mySecret is the token.

and created this function on the backend bigquery.jsw file

// Imports the Google Cloud client library
import {BigQuery} from '@google-cloud/bigquery';
import _ from 'lodash';
import {getSecret} from 'wix-secrets-backend';

//...


//POST https://bigquery.googleapis.com/bigquery/v2/projects/{projectId}/datasets/{datasetId}/tables/{tableId}/insertAll

// Copyright 2019 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//      http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

'use strict';

  const datasetId = 'wix_data';
  const tableId = 'user_data';
  const projectId = 'XXXXholder-XXXX'
  const current_timestamp = new Date().getTime()
  

  
 

      export async function insertRowsAsStream() {
    
      const mySecret = await getSecret("google_cloud_api");
      
      const bigquery = new BigQuery({ projectId, mySecret });
    
        // Inserts the JSON objects into my_dataset:my_table.
    
        /**
         * TODO(developer): Uncomment the following lines before running the sample.
         */
        
        const rows = [
          {timestamp: current_timestamp, token: 'test'},
         // {name: 'Jane', age: 32},
        ];
    
        // Insert data into a table
        return await bigquery
          .dataset(datasetId)
          .table(tableId)
          .insert(rows);
       
      }
    
    //main(...process.argv.slice(2));

*getSecret is google console JSON token, get from the secret manager

I called this function from Wix page and nothing happened.

Errors : UPDATE : adding {} to import BigQuery from '@google-cloud/bigquery';

and now the errors not constractor error , but QAuth2 erorrs: enter image description here

enter image description here


Solution

  • I changed the constractor to be like this:

    const bigquery = await new BigQuery({ projectId: projectId, keyFilename: path });
    

    and fixed the timestamp expression
    const timestamp = BigQuery.timestamp(new Date());