Passing files from Google Cloud Container Builder to Docker build task

Context

A Ruby on Rails application built with Container Builder destined for App Engine. We require bundler to be able to install dependencies from a private git repository using a SSH key.

The SSH Keys come from a secure bucket where they pass through KMS for decryption. Those steps are fine. However, the final step to build the container with Docker fails nothing able to access the SSH key.

I do not have any previous extensive experience with Docker so I assume this is a simple issue.

cloudbuild.yml

steps:
  # Get and prepare Deploy Key
- name: 'gcr.io/cloud-builders/gsutil'
  args: ['cp', 'gs://[PROJECT-BUCKET]/git_id_rsa.enc', '/root/.ssh/git_id_rsa.enc']
  volumes:
  - name: 'ssh-setup'
    path: /root/.ssh
- name: 'gcr.io/cloud-builders/gcloud'
  args:
  - kms
  - decrypt
  - --ciphertext-file=/root/.ssh/git_id_rsa.enc
  - --plaintext-file=/root/.ssh/git_id_rsa
  - --location=global
  - --keyring=[KEYRING]
  - --key=[KEY]
  volumes:
  - name: 'ssh-setup'
    path: /root/.ssh
- name: 'gcr.io/cloud-builders/gcloud'
  entrypoint: /workspace/deploy/git-prepare.sh
  volumes:
  - name: 'ssh-setup'
    path: /root/.ssh
  # ... Omitted steps ...
  # Docker build
- name: 'gcr.io/cloud-builders/docker'
  args: ['build', '-t', 'gcr.io/$PROJECT_ID/[PROJECT-NAME]', '.']
  volumes:
  - name: 'ssh-setup'
    path: /root/.ssh
images: ['gcr.io/$PROJECT_ID/[PROJECT-NAME]']

some identifiers have been omitted

deploy/git-prepare.sh — This performs some shell commands to get the ssh directory refilled with necessary information.

#!/bin/bash

mkdir -p /root/.ssh

touch /root/.ssh/config
ssh-keyscan bitbucket.org >> /root/.ssh/known_hosts

touch /root/.ssh/config
echo -e "\nHost bitbucket.org\n    IdentityFile /root/.ssh/git_id_rsa" >> /root/.ssh/config

if [ -f /root/.ssh/git_id_rsa.enc ]; then
    rm /root/.ssh/git_id_rsa.enc
fi

Dockerfile

# .. OMITTING BOILERPLATE FOR RAILS APP SETUP ...
# Copy the application files.
COPY . /app/

# Copy the SSH keys and config for bundler
VOLUME /root/.ssh

COPY /root/.ssh/known_hosts ~/.ssh/known_hosts
COPY /root/.ssh/config ~/.ssh/config
COPY /root/.ssh/git_id_rsa ~/.ssh/git_id_rsa

Problem:

The build task (run using a build trigger) fails with:

...omitted lines above...   
Step #5: COPY failed: stat /var/lib/docker/tmp/docker-builderxxxxxxxxx/root/.ssh/known_hosts: no such file or directory
Step #5: Step 7/14 : COPY /root/.ssh/known_hosts ~/.ssh/known_hosts

I have a feeling I don't grasp the way that Container Builder and Docker share volumes and data.

Solution

Dockerfile COPY can use multiple <src> resources, but the paths of files and directories will be interpreted as relative to the source of the context of the build.

That is your current path where you execute the docker build . command.

In your case, if /root/.ssh is mounted when the Dockerfile executes its step, a simple RUN cp /root/.ssh/... /destination/path would be enough.

However, you cannot mount a volume at docker build time (see moby issue 14080), so check this solution: a multi-stage build can help.