Search code examples
gitdockergitlabgitlab-cidocker-in-docker

GitLab CI - Trying to use docker buildx to build for ARM64


Trying to use docker buildx with GitLabs shared runners to build a Docker image that can be run on my Raspberry Pi. Job fails saying git is not in PATH but git is installed in image: docker:stable-git. Any known fixes or a better way to build an ARM64-compatible image without having to build on the Raspberry Pi itself? (RPi becomes unusable while building on it due to CPU usage)

deploy:
  image: docker:stable-git
  stage: deploy
  services:
    - docker:dind
  before_script:
    - export DOCKER_BUILDKIT=1
    - export DOCKER_CLI_EXPERIMENTAL=enabled
    - docker build --platform=local -o . git://github.com/docker/buildx
    - mv buildx ~/.docker/cli-plugins/docker-buildx
  script:
    - docker buildx build --platform linux/amd64,linux/arm64 -t $CI_REGISTRY_IMAGE .
    - docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $CI_REGISTRY
    - docker push $CI_REGISTRY_IMAGE
$ export DOCKER_BUILDKIT=1
$ export DOCKER_CLI_EXPERIMENTAL=enabled
$ docker build --platform=local -o . git://github.com/docker/buildx
#1 [internal] load git source git://github.com/docker/buildx
#1 ERROR: failed to init repo at /var/lib/docker/overlay2/xxx/diff: exec: "git": executable file not found in $PATH
------
 > [internal] load git source git://github.com/docker/buildx:
------
failed to solve with frontend dockerfile.v0: failed to resolve dockerfile: failed to build LLB: failed to load cache key: failed to init repo at /var/lib/docker/overlay2/xxx/diff: exec: "git": executable file not found in $PATH
ERROR: Job failed: exit code 1

Solution

  • It looks like the issue was that the build was failing due to git not being installed in docker:dind

    Was able to work around the issue by cloning Docker/BuildX in a separate stage, running docker build on it, then exporting it to the deploy stage using artifacts.

    buildx:
      image: docker:19.03-git
      stage: buildx
      variables:
        GIT_STRATEGY: none
      artifacts:
        paths:
          - buildx
        expire_in: 1 hour
      services:
        - docker:19.03-dind
      script:
        - export DOCKER_BUILDKIT=1
        - git clone git://github.com/docker/buildx ./docker-buildx
        - docker build --platform=local -o . ./docker-buildx
    
    deploy:
      image: docker:19.03
      stage: deploy
      services:
        - name: docker:19.03-dind
          command: ["--experimental"]
      before_script:
        - mkdir -p ~/.docker/cli-plugins
        - mv buildx ~/.docker/cli-plugins/docker-buildx
        - docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
      script:
        - docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $CI_REGISTRY
        - docker buildx create --use --name mybuilder
        - docker buildx build --platform linux/amd64,linux/arm64,linux/arm/v7 --push -t $CI_REGISTRY_IMAGE .
    

    I've decided to update my answer with info about using BuildX in CircleCI due to changes made by GitLab to free accounts as well as comments posted by j0nl1 and Yajo

    You can download the BuildX binary and use it in your CircleCI job by doing the following

    jobs:
      build-and-push:
        executor: docker/machine
        steps:
          - checkout
          - run:
              name: Remove old installation
              command: |
                sudo systemctl stop docker.service
                sudo apt remove docker-ce docker-ce-cli containerd.io
          - docker/install-docker:
              install-dir: /usr/bin
          - run:
              name: Restart docker daemon with experimental features
              command: |
                sudo bash -c 'echo "{\"experimental\":true}" > /etc/docker/daemon.json'
                if [[ $EUID == 0 ]]; then export SUDO=""; else export SUDO="sudo"; fi
                $SUDO systemctl unmask docker.service
                $SUDO systemctl unmask docker.socket
                $SUDO systemctl start docker.service
          - run:
              name: Install buildx
              command: |
                BUILDX_VERSION="v0.4.1"
    
                curl -sSLo docker-buildx "https://github.com/docker/buildx/releases/download/$BUILDX_VERSION/buildx-$BUILDX_VERSION.linux-amd64"
                chmod a+x docker-buildx
                mkdir -p ~/.docker/cli-plugins
                mv docker-buildx ~/.docker/cli-plugins/docker-buildx
    
                docker version
                docker buildx install
          - docker/check:
              registry: $DOCKER_REGISTRY
          - run:
              name: Create buildx profile
              command: |
                docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
                docker buildx create --use --name mybuilder
          - docker/build:
              registry: $DOCKER_REGISTRY
              image: username/repo
              tag: latest
              extra_build_args: --platform linux/amd64,linux/arm64,linux/arm/v7 --push