JWT Authentication in Google Cloud Functions

Question

I'm having trouble troubleshooting the cause for a 403 response from the Google Dataflow API while called using the module "googleapis" inside a Google Cloud Function.

The code works when run on my PC using the same code that is being run on Cloud Functions. The JWT .json file is being retrieved from an object stored on a Google Storage bucket.

The code looks like this:

...
return getToken(). //Retrieves the JWT Client from Google Storage
      then(function (jwtToken) {
        console.log("Token: ", JSON.stringify(jwtToken));
        return dataFlowList({
          projectId: adc.projectId,
          auth: jwtToken,
          filter: "TERMINATED"
        }).then(list => filterDataflowJobList(list))
...

Here the getToken function:

...
let storage: CloudStorage.Storage = CloudStorage({
  projectId: adc.projectId
});
var bucket: CloudStorage.Bucket = storage.bucket(bucketName);

var bucketGetFiles = PromiseLab.denodeify(bucket.getFiles);

var stream = bucket.file(jwtJsonFileName).createReadStream();
return toString(stream)
  .then(function (msg) {
    var jsonJwt = JSON.parse(msg);
    var jwtClient = new google.auth.JWT(
      jsonJwt.client_email,
      null,
      jsonJwt.private_key,
      ['https://www.googleapis.com/auth/cloud-platform'], // an array of auth scopes
      null
    );
    return jwtClient;
  }).catch(function (error) {
    console.log("Error while trying to retrieve JWT json");
    throw error;
  })
}
...

I'm based in EU and Cloud Functions are US-bound, could that be the case? Dataflow jobs are also run in US

Answer 1

While running on Google Function, the authentication retrieval method I'm using is not retrieving the projectId, hence the unauthorized.

async function getADC() {
  // Acquire a client and the projectId based on the environment. This method looks
  // for the GCLOUD_PROJECT and GOOGLE_APPLICATION_CREDENTIALS environment variables.
  const res = await auth.getApplicationDefault();
  let client = res.credential;

  // The createScopedRequired method returns true when running on GAE or a local developer
  // machine. In that case, the desired scopes must be passed in manually. When the code is
  // running in GCE or a Managed VM, the scopes are pulled from the GCE metadata server.
  // See https://cloud.google.com/compute/docs/authentication for more information.
  if (client.createScopedRequired && client.createScopedRequired()) {
    // Scopes can be specified either as an array or as a single, space-delimited string.
    const scopes = ['https://www.googleapis.com/auth/cloud-platform'];
    client = client.createScoped(scopes);
  }
  return {
    client: client,
    projectId: res.projectId
  }
}

I discovered it by looking at the Header request in the error log, it was in the form of: url: 'https://dataflow.googleapis.com/v1b3/projects//jobs' (notice the double "//" between projects and jobs.