How do I pass custom GoogleAuth credentials to cloud-sql-connector using environment variables

I want to connect my Vercel hosted backend to my Google Cloud SQL instance using @google-cloud/cloud-sql-connector and service account credentials.

At the moment I can establish a connection in my local development when I set the GOOGLE_APPLICATION_CREDENTIALS environment variable to the path where the service account credentials are located.

However, I want to supersede the GOOGLE_APPLICATION_CREDENTIALS and pass in custom credentials for the service account. Vercel doesn't support filepaths so I need to add credentials from environment variables.

I have tried different options however the Connector() is not using the credentials from the GoogleAuth object and is still looking for the default credentials.

How can I get the connector to use the credentials in the GoogleAuth object?

I have tried specifying the actual JSON data (using an environment variable) in credentials as follows:

const connector = new Connector({
    auth: new GoogleAuth({
        credentials: process.env.CREDENTIALS_JSON,
        scopes: ['https://www.googleapis.com/auth/cloud-platform'],
      })
  });

and even tried specifying the path to the keyFile as follows (when testing locally):

 const connector = new Connector({
    auth: new GoogleAuth({
        keyFile: 'certificates/survey-engine-301112-d65599b32daa.json',
        scopes: ['https://www.googleapis.com/auth/cloud-platform'],
      }),
  });

However both ways still give:

Error: Could not load the default credentials

ATTEMPT 2 - 2023-12-06

I can create a GoogleAuth object from the service account JSON stored in environment variables (as per this documentation) however I am trying to work out how to have the GoogleAuth.fromJSON(keys) work with cloud-sql-connector as I am still not getting a connection.

It is not throwing an error but it is not connecting either.

Here is my current code:

import mysql from 'mysql2/promise';
import {Connector} from '@google-cloud/cloud-sql-connector';
import {GoogleAuth} from 'google-auth-library';

let pool;

// load the environment variable with our keys
const keysEnvVar = process.env.CREDENTIALS_JSON
if (!keysEnvVar) {
  throw new Error('The CREDENTIALS_JSON environment variable was not found!');
}
const keys = JSON.parse(keysEnvVar);
console.log("keys=",keys)

const connector = new Connector({
auth: new GoogleAuth.fromJSON(keys),
});

const clientOpts = await connector.getOptions({
instanceConnectionName: 'survey-engine-301112:australia-southeast2:database-australia',
ipType: 'PUBLIC',
scope: ['https://www.googleapis.com/auth/cloud-platform'],
});

if (!pool){
pool = await mysql.createPool({
...clientOpts,
user: process.env.DBUSER,
password: process.env.DBPASSWORD,
});
}

const conn = await pool.getConnection();
const [result] = await conn.query( `SELECT NOW();`);
console.table(result); // prints returned time value from server
conn.release();

export default pool

Here is my output from console.log("keys=",keys). I have hashed out some of the details.

keys= {
  type: 'service_account',
  project_id: 'survey-######-30####',
  private_key_id: 'd65############e9f',
  private_key: '-----BEGIN PRIVATE KEY-----\n' +
    'MIIEvwIBADANBgkqhkiG9w0BAQEFAASCBKkwggSlAgEAAoIBAQDkYpkZMXRJQfp+\n' +
    '#########################\n'+    
    'cnTc7caSvsgRFpeLCMxV5REeQw==\n' +
    '-----END PRIVATE KEY-----\n',
  client_email: '[email protected]',
  client_id: '10#################22',
  auth_uri: 'https://accounts.google.com/o/oauth2/auth',
  token_uri: 'https://oauth2.googleapis.com/token',
  auth_provider_x509_cert_url: 'https://www.googleapis.com/oauth2/v1/certs',
  client_x509_cert_url: 'https://www.googleapis.com/robot/v1/metadata/x509/backend-to-db%40survey-engine-301112.iam.gserviceaccount.com',
  universe_domain: 'googleapis.com'
}

I am using cloud-sql-connector version 1.2.0, google-auth-library version 9.2.0, mysql2 version 3.6.2 and node version 20.10.0

Solution

The below example showcases how to initialize a Cloud SQL Node.js Connector Connector using a custom GoogleAuth client from a service account key that is loaded into an environment variable.

Note: Keys are expected to be loaded into env variable as a string... export CREDS='{...}' and not direct JSON.

import {Connector} from '@google-cloud/cloud-sql-connector';
import {GoogleAuth} from 'google-auth-library';

// load the environment variable with our keys
const keysEnvVar = process.env.CREDS
if (!keysEnvVar) {
  throw new Error('The $CREDS environment variable was not found!');
}
const keys = JSON.parse(keysEnvVar);

const auth = new GoogleAuth({
    scopes: ['https://www.googleapis.com/auth/sqlservice.admin']
});

const connector = new Connector({
    auth: auth.fromJSON(keys),
});