Could not start backup:Request failed with status code 400 Google Cloud

Question

I'm trying to create a backup system for Firestore. I followed every step of this guide and when I tried to deploy the code, it returned Request failed with status code 400

PROJECT-ID@appspot.gserviceaccount.com permissions: Cloud Datastore Import Export Admin, Editor, Storage Admin

This is the code of app.js

'use strict';


const axios = require('axios');
const dateformat = require('dateformat');
const { google } = require('googleapis');
const express = require('express');
const util = require('util')
const request = require('request');
const admin = require('firebase-admin');

const {Storage} = require('@google-cloud/storage');

// Creates a client
const storage = new Storage();


admin.initializeApp({
  credential: admin.credential.applicationDefault()
});

const db = admin.firestore();

const googleMapsClient = require('@google/maps').createClient({
  key: 'AIza*****',
  Promise: Promise
});

const app = express();

// Trigger a backup
app.get('/cloud-firestore-export', async (req, res) => {
  const auth = await google.auth.getClient({
    scopes: ['https://www.googleapis.com/auth/datastore'],
  });

  const accessTokenResponse = await auth.getAccessToken();
  const accessToken = accessTokenResponse.token;

  const headers = {
    'Content-Type': 'application/json',
    Authorization: 'Bearer ' + accessToken,
  };

  const { outputUriPrefix } = req.query;
  if (!outputUriPrefix) {
    res.status(500).send('outputUriPrefix required');
  } else if (outputUriPrefix && outputUriPrefix.indexOf('gs://') !== 0) {
    res.status(500).send('Malformed outputUriPrefix: ${outputUriPrefix}');
  }

  // Construct a backup path folder based on the timestamp
  const timestamp = dateformat(Date.now(), 'yyyy-mm-dd-HH-MM-ss');
  let path = outputUriPrefix;
  if (path.endsWith('/')) {
    path += timestamp;
  } else {
    path += '/' + timestamp;
  }

  const body = {
    outputUriPrefix: path,
  };

  // If specified, mark specific collections for backup
  const { collections } = req.query;
  if (collections) {
    body.collectionIds = collections.split(',');
  }

  const projectId = process.env.GOOGLE_CLOUD_PROJECT;
  const url = 'https://firestore.googleapis.com/v1beta1/projects/' + projectId + '/databases/(default):exportDocuments';

  try {
    const response = await axios.post(url, body, { headers });
    res
      .status(200)
      .send(response.data)
      .end();
  } catch (e) {
    if (e.response) {
      console.warn(e.response.data);
    }

    res
      .status(500)
      .send('Could not start backup:' + e.message)
      .end();
  }
});


°°°°
// Start the server
const PORT = process.env.PORT || 8080;
app.listen(PORT, () => {
  console.log('App listening on port ${PORT}');
  console.log('Press Ctrl+C to quit.');
});

I have another function that is listening to '/'. Is possible that this can cause the problem?

package.json:

{
  "name": "solution-scheduled-backups",
   "version": "1.0.0",
  "description": "Scheduled Cloud Firestore backups via AppEngine cron",
  "main": "app.js",
  "engines": {
    "node": "10.x.x"
  },
  "scripts": {
    "deploy": "gcloud app deploy --quiet app.yaml cron.yaml",
    "start": "node app.js"
  },
  "author": "Google, Inc.",
  "license": "Apache-2.0",
  "dependencies": {
    "@google-cloud/storage": "^3.2.1",
    "@google/maps": "^0.5.5",
    "axios": "^0.19.0",
    "dateformat": "^3.0.3",
    "express": "^4.17.1",
    "firebase-admin": "^8.4.0",
    "googleapis": "^42.0.0",
    "request": "^2.88.0"
  },
  "devDependencies": {
    "prettier": "^1.18.2"
  }
}

I also look inside Cron log and there is nothing related to the error. It only returns 500 error

Answer 1

The main problem is the bucket location

When you create a backup bucket you must use Multi-Region or you will receive a deny from the server.

I think this is a bug of Google Cloud

Solution

Delete the bucket and create a new one with a Multi-Regional location

The error with Single Location:

'Bucket backup-bucket is in location EUR4. This project can only operate on buckets 
spanning location europe-north1 or europe-west1 or eu or europe-west2 or 
europe-west3 or europe-west4 or europe-west5 or europe-west6.',