I'm trying to build a custom container with Buildah via a Dockerfile that will run some tasks in Celery, but the tasks need access to a library available in a private repository on our local Gitlab instance. It works if I copy the library from a directory I cloned locally, but it would be best if I could just clone a copy to the container in the Dockerfile. However, I can't get the git clone to work inside the Dockerfile when trying to build it in Buildah. It doesn't seem to be able to read my SSH keys, which are stored on the host at ~/.ssh/id_rsa. I'm trying to follow this from the Buildah man page:
--ssh=default|id[=socket>|<key>[,<key>]
SSH agent socket or keys to expose to the build. The socket path can be left empty to use the
value of default=$SSH_AUTH_SOCK
To later use the ssh agent, use the --mount flag in a RUN instruction within a Containerfile:
RUN --mount=type=secret,id=id mycmd
So in my Dockerfile:
RUN mkdir -p -m 0700 ~/.ssh && ssh-keyscan -t ed25519 gitlab.mycompany.com >> ~/.ssh/known_hosts
RUN --mount=type=ssh git clone [email protected]:jdoe/library.git /opt/library
And when I try to build it in Builad:
buildah build --ssh=default -f celery/Dockerfile -t celery
And the error when Buildah gets to the step where it's trying to clone the git repository:
Permission denied, please try again.
Permission denied, please try again.
[email protected]: Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
error building at STEP "RUN --mount=type=ssh git clone [email protected]:jdoe/library.git /opt/library": error while running runtime: exit status 128
Finished
git clones work correctly using my default SSH keys on my host, but whatever I'm doing to access the keys when building the Dockerfile in Buildah isn't working correctly. What do I need to change to get use the SSH keys inside of Buildah?
PS Buildah version, on RHEL8:
$ buildah -v
buildah version 1.26.2 (image-spec 1.0.2-dev, runtime-spec 1.0.2-dev)
EDIT: So I figured out how to get it to work via the --secret flag. Dockerfile:
RUN --mount=type=secret,id=id_rsa GIT_SSH_COMMAND="ssh -i /run/secrets/id_rsa" git clone [email protected]:jdoe/library.git /opt/library
Command line:
buildah build --secret id=id_rsa,src=/home/wile_e8/.ssh/id_rsa -f celery/Dockerfile -t celery
This works, although only once. When I try to run this command next in the Dockerfile:
WORKDIR /opt/library
RUN --mount=type=secret,id=id_rsa GIT_SSH_COMMAND="ssh -i /run/secrets/id_rsa" git fetch --all --tags --prune
I get the following error:
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@ WARNING: UNPROTECTED PRIVATE KEY FILE! @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
Permissions 0755 for '/run/secrets/id_rsa' are too open.
It is required that your private key files are NOT accessible by others.
This private key will be ignored.
Load key "/run/secrets/id_rsa": bad permissions
Permission denied, please try again.
Permission denied, please try again.
[email protected]: Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
Looks like I'll have to figure out how to set permissions on the secret file. But I still have no idea on how to get the --ssh flag to work correctly, which should be easier than doing all this stuff with the secret file.
EDIT 2: And here is how I managed to run multiple commands that contact the private Gitlab repository - Dockerfile:
ENV GIT_SSH_COMMAND="ssh -i /run/secrets/id_rsa"
RUN --mount=type=secret,id=id_rsa git clone [email protected]:jdoe/library.git /opt/library && \
cd /opt/library && \
git fetch --all --tags --prune && \
git checkout tags/1.0.0 -b 1.0.0
Still not as convenient as figuring out the correct syntax for the --ssh flag, but it works.
I eventually figured out how to format this to get the --ssh flag to work. Although I'm now updated to version 1.27.2, so maybe it was a bug fix.
$ buildah -v
buildah version 1.27.2 (image-spec 1.0.2-dev, runtime-spec 1.0.2-dev)
But here is how I formatted the buildah command:
buildah build --ssh id=/home/wile_e8/.ssh/id_rsa -f celery/Dockerfile -t celery
And here is the git fetch line in the Dockerfile:
RUN --mount=type=ssh,id=id git clone [email protected]:jdoe/library.git /opt/library && \
cd /opt/library && \
git fetch --all --tags --prune && \
git checkout tags/1.0.0 -b 1.0.0
I don't know why --ssh=default doesn't automatically pull ~/.ssh/id_rsa, but manually specifying that file in this way works.