How to pass substitution variables on Google Cloud build to run a dockerfile

Question

I have set up a Github trigger on Google Cloud build. I already have a dockerfile to do the build steps. Since there is no provision to pass substitution variables I am trying to build it with cloud build configuration file (yaml) and then passing the path to the dockerfile in the configuration file.

This is the cloud build configuration file where I need to pass 5 variables for the Dockerfile to consume and the yaml file goes like this:

steps:
- name: 'gcr.io/cloud-builders/docker'
  env:
  - 'ACCESS_TOKEN=$ACCESS_TOKEN'
  - 'BRANCH=$BRANCH'
  - 'UTIL_BRANCH=$UTIL_BRANCH'
  - 'ARG_ENVIRONMENT=$ARG_ENVIRONMENT'
  - 'ARG_PYTHON_SCRIPT=$ARG_PYTHON_SCRIPT'
  args:
  - build
  - "--tag=gcr.io/$PROJECT_ID/quickstart-image"
  - "--file=./twitter/dax/processing_scripts/Dockerfile"
  - .

When the trigger runs the build, I get an error in one of the build steps in dockerfile saying that the variable is not available. It's clear that the environment variable passed in the yaml file is not being passed to the Dockerfile for consumption. And this is how the I have filled the substitution variables in on the trigger page

Pasting the build code from step 5 where the error occurs, before which is just apt-get update commands running:

Step 5/28 : RUN git config --global url."https://${ACCESS_TOKEN}:@github.com/".insteadOf "https://github.com/" && echo $(ACCESS_TOKEN)
 ---> Running in f7b94bc2a0d9

/bin/sh: 1: ACCESS_TOKEN: not found
Removing intermediate container f7b94bc2a0d9
 ---> 30965207dcec
Step 6/28 : ARG BRANCH
 ---> Running in 93e36589ac48
Removing intermediate container 93e36589ac48
 ---> 1d1508b1c1d9
Step 7/28 : RUN git clone https://github.com/my_repo45/twitter.git -b "${BRANCH}"
 ---> Running in fbeb93dbb113
Cloning into 'twitter'...
remote: Repository not found.
fatal: Authentication failed for 'https://github.com/my_repo45/twitter.git/'
The command '/bin/sh -c git clone https://github.com/my_repo45/twitter.git -b "${BRANCH}"' returned a non-zero code: 128
ERROR
ERROR: build step 0 "gcr.io/cloud-builders/docker" failed: step exited with non-zero status: 128```

Could anyone please point out what the issue and validate if the environment declaration is proper?

Answer 1

You would still need to add the variables to the docker build command. As in your current situation, the substitution variables are only available for you within cloud build and not as a regular environment variable. A simplified example of how to pass those variables to cloud build is outlined below. Note that you can only use this with ARG in your dockerfile, not with ENV.

steps:
- name: 'gcr.io/cloud-builders/docker'
  env:
    - 'ACCESS_TOKEN=$_ACCESS_TOKEN'
  args:
    - build
    - "--build-arg=ACCESS_TOKEN=${ACCESS_TOKEN}"
    - "--tag=gcr.io/$PROJECT_ID/quickstart-image"
    - "--file=./twitter/dax/processing_scripts/Dockerfile"

Another option is to export the environment variable within the same build step your docker build command is in:

steps:
- name: 'gcr.io/cloud-builders/docker'
  entrypoint: 'bash'
  env:
    - 'ACCESS_TOKEN=$_ACCESS_TOKEN'
  args:
  - '-c'
  - |
    export ACCESS_TOKEN=${ACCESS_TOKEN}
    docker build \
      --tag=gcr.io/$PROJECT_ID/quickstart-image" \
      --file=./twitter/dax/processing_scripts/Dockerfile"

Of course, you could argue if the env field is still needed in this setup, I'll leave that up to you.