Search code examples
dockergithub-actionsbuildx

Github docker/build-push-action with multiple architectures is slow

I am building a docker image as a step in my CD in github actions and I recently enabled multiple architecture builds because I am on an M2 Mac and so I would like arm builds as well. I noticed that after enabling that my build times sky-rocketed.

Am I doing something wrong or is this to be expected?

Step:

# Build and push Docker image with Buildx (don't push on PR)
# https://github.com/docker/build-push-action
- name: Build and push Docker image
  id: build-and-push
  uses: docker/[email protected]
  with:
    context: ./src
    push: ${{ github.event_name != 'pull_request' }}
    tags: ${{ steps.meta.outputs.tags }}
    labels: ${{ steps.meta.outputs.labels }}
    platforms: linux/amd64,linux/arm64
    cache-from: type=gha
    cache-to: type=gha,mode=max

I do also notice that I see these messages now:

#19 7.855 warning Pattern ["string-width@^4.1.0"] is trying to unpack in the same destination "/usr/local/share/.cache/yarn/v6/npm-string-width-cjs-4.2.3-269c7117d27b05ad2e536830a8ec895ef9c6d010-integrity/node_modules/string-width-cjs" as pattern ["string-width-cjs@npm:string-width@^4.2.0"]. This could result in non-deterministic behavior, skipping.
Solution

  • Building Docker images for multiple architectures can indeed increase the build time, especially if you're doing it on a single machine. This is because Docker has to build a separate image for each architecture, which can be time-consuming.

    When you build for multiple architectures, Docker uses QEMU to emulate the target architecture. This emulation process can be slower than native builds, especially for complex applications.

    If the increased build time is a problem, you might consider a few options:

    • Parallelize the builds: If you have access to multiple machines, you could build the images for different architectures in parallel.

    • Use buildx bake: Docker's buildx tool has a bake command that can build images for multiple platforms in parallel, even on a single machine. This can potentially reduce the build time.

    • Use a cloud-based build service: GitHub Actions does indeed support parallelism, but it depends on how your workflow is set up. If you're building Docker images for multiple architectures in a single job, then those builds are likely happening sequentially, which could explain the increased build time.

    To take advantage of GitHub Actions' parallelism, you could split the builds into separate jobs and use a build matrix to specify the different architectures. Here's an example of how you might set this up:

    name: Build Docker Images

on: [push, pull_request]

jobs:
  build:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        architecture: [amd64, arm64]
    steps:
    - uses: actions/checkout@v2

    - name: Set up Docker Buildx
      uses: docker/setup-buildx-action@v1

    - name: Build and push Docker image
      uses: docker/build-push-action@v2
      with:
        context: .
        push: true
        tags: user/app:${{ matrix.architecture }}
        platforms: linux/${{ matrix.architecture }}

    The parallelism in this GitHub Actions workflow is determined by the strategy.matrix key.

    solving lowercase github.repository_owner:

    This is how you can run a shell script to convert your username to a lowercase env variable.

    - name: Lowercase GitHub username
  run: echo "DOCKER_USERNAME=$(echo ${{ github.repository_owner }} | tr '[:upper:]' '[:lower:]')" >> $GITHUB_ENV

- name:
  ...
  tags: ${{ env.DOCKER_USERNAME }}/your-image-name:${{ steps.meta.outputs.tags }}

    In GitHub Actions, the GITHUB_ENV is a special environment file. When you write a string to this file in the format NAME=VALUE. The env context is a context available in GitHub Actions that contains the environment variables available to the runner. When you write to the GITHUB_ENV file, the environment variables you set are added to this context.