Gcloud build trigger environment variable substitution in app.yaml for appEngine

Question

I am trying to substitue variable in app.yaml with a cloud build trigger.

I Added substitution variable in build trigger.

Add environment variables to app.yaml in a way they can be easily substituted with build trigger variables. Like this:

env_variables:
 SECRET_KEY: %SECRET_KEY%

Add a step in cloudbuild.yaml to substitute all %XXX% variables inside app.yaml with their values from build trigger.

    steps:
    - name: node:10.15.1
       entrypoint: npm
       args: ["install"]
    - name: 'gcr.io/cloud-builders/gcloud'
       entrypoint: bash
       args:
         - '-c'
         - |
           sed -i 's/%SESSION_SECRET%/'${_SESSION_SECRET}'/g' app.yaml
    timeout: "1600s"

The problem is that Gcloud Build throw an exception :

Already have image (with digest): gcr.io/cloud-builders/gcloud
bash: _L/g: No such file or directory

Why ? How can I make a substitution of my app.yaml ?

I have a app.yaml to the root of the project at the same level of the cloudbuild.yaml

UPDATED

I am trying to build and debug gcloud locally with this command:

sudo cloud-build-local --config=cloudbuild.yaml --write-workspace=../workspace --dryrun=false --substitutions=_SESSION_SECRET=test --push .

When I take a look into the app.yaml file, the substitution worked as expected and there is no exception at all.

What is the difference with the gcloud build environment ?

Answer 1

OK I finally decided to use github action instead of google cloud triggers.

Since Google cloud triggers aren't able to find its own app.yaml and manage the freaking environment variable by itself.

Here is how to do it:

My environment : App engine, standard (not flex), Nodejs Express application, a PostgreSQL CloudSql

First the setup :

1. Create a new Google Cloud Project (or select an existing project).

2. Initialize your App Engine app with your project.

[Create a Google Cloud service account][sa] or select an existing one.

3. Add the the following Cloud IAM roles to your service account:

    App Engine Admin - allows for the creation of new App Engine apps

    Service Account User - required to deploy to App Engine as service account

    Storage Admin - allows upload of source code

    Cloud Build Editor - allows building of source code

[Download a JSON service account key][create-key] for the service account.

4. Add the following [secrets to your repository's secrets][gh-secret]:

    GCP_PROJECT: Google Cloud project ID

    GCP_SA_KEY: the downloaded service account key

The app.yaml

runtime: nodejs14
env: standard
env_variables:
  SESSION_SECRET: $SESSION_SECRET
beta_settings:
  cloud_sql_instances: SQL_INSTANCE

Then the github action

name: Build and Deploy to GKE

on: push

env:
  PROJECT_ID: ${{ secrets.GKE_PROJECT }}
  DATABASE_URL: ${{ secrets.DATABASE_URL}}
jobs:
  setup-build-publish-deploy:
    name: Setup, Build, Publish, and Deploy
    runs-on: ubuntu-latest

steps:
 - uses: actions/checkout@v2
 - uses: actions/setup-node@v2
   with:
    node-version: '12'
 - run: npm install
 - uses: actions/checkout@v1
 - uses: ikuanyshbekov/app-yaml-env-compiler@v1.0
   env:
    SESSION_SECRET: ${{ secrets.SESSION_SECRET }}  
 - shell: bash
   run: |
        sed -i 's/SQL_INSTANCE/'${{secrets.DATABASE_URL}}'/g' app.yaml
 - uses: actions-hub/gcloud@master
   env:
    PROJECT_ID: ${{ secrets.GKE_PROJECT }}
    APPLICATION_CREDENTIALS: ${{ secrets.GCLOUD_AUTH }}
    CLOUDSDK_CORE_DISABLE_PROMPTS: 1
   with:
    args: app deploy app.yaml

To add secrets into git hub action you must go to : Settings/secrets

Take note that I could handle all the substitution with the bash script. So I would not depend on the github project "ikuanyshbekov/app-yaml-env-compiler@v1.0"

It's a shame that GAE doesn't offer an easiest way to handle environment variable for the app.yaml. I don't want to use KMS since I need to update the beta-settings/cloud sql instance.. I really needed to substitute everything into the app.yaml.

This way I can make a specific action for the right environment and manage the secrets.