Search code examples
.netlinuxdockerkubernetesarm64

"exec format error" running arm64 docker image using dotnet


I am trying to build and run docker image with dotnet on arm64 architecture. I can see that the image was properly built as a MULTI-PLATFORM, see this Docker Hub image. It contains both amd64 and arm64.

I have a K8s cluster that is running on arm64, so I expected this should be compatible. However, running this image gives me:

exec /opt/czertainly/entry.sh: exec format error

which seems to be wrong architecture according to multiple sources. I am not able to figure out what is wrong.

This is the Dockerfile:

#See https://aka.ms/containerfastmode to understand how Visual Studio uses this Dockerfile to build your images for faster debugging.

FROM --platform=$BUILDPLATFORM mcr.microsoft.com/dotnet/aspnet:8.0 AS base
ARG TARGETARCH
WORKDIR /app

FROM --platform=$BUILDPLATFORM mcr.microsoft.com/dotnet/sdk:8.0 AS build
ARG TARGETARCH
WORKDIR /
COPY ["src/Czertainly.Auth/Czertainly.Auth.csproj", "Czertainly.Auth/"]
RUN dotnet restore "Czertainly.Auth/Czertainly.Auth.csproj" -a $TARGETARCH
COPY . .
WORKDIR "/src/Czertainly.Auth"
RUN dotnet build "Czertainly.Auth.csproj" -c Release -o /app/build -a $TARGETARCH

FROM build AS publish
ARG TARGETARCH
RUN dotnet publish "Czertainly.Auth.csproj" -c Release -o /app/publish -a $TARGETARCH

FROM base AS final
ARG TARGETARCH

MAINTAINER CZERTAINLY <[email protected]>

RUN addgroup --system --gid 10001 czertainly && adduser --system --home /opt/czertainly --uid 10001 --ingroup czertainly czertainly
#RUN addgroup --group czertainly --gid 10001 && adduser --uid 10001 --gid 10001 "czertainly" 

COPY --from=publish /app/publish /opt/czertainly
COPY ./docker /opt/czertainly

WORKDIR /opt/czertainly

ENV COMPlus_EnableDiagnostics=0

ENV AUTH_DB_CONNECTION_STRING=
ENV AUTH_CREATE_UNKNOWN_USERS=false
ENV AUTH_CREATE_UNKNOWN_ROLES=false

USER 10001

ENTRYPOINT ["/opt/czertainly/entry.sh"]

Can it be connected somehow with the entry.sh? This is the content of entry.sh, it is ok from my perspective:

#!/bin/bash

czertainlyHome="/opt/czertainly"
source ${czertainlyHome}/static-functions

log "INFO" "Launching the Auth service"
dotnet Czertainly.Auth.dll

Any clue what can be wrong?

Build is executed using GitHub action:

name: Publish Docker image

on:
  push:
    branches: [develop]
    tags:
      - '*'
  workflow_dispatch:

jobs:
  push_to_registry:
    name: Push Docker images
    runs-on: ubuntu-latest
    steps:
      - name: Check out the repo
        uses: actions/checkout@v4

      - name: Install Cosign
        uses: sigstore/[email protected]

      - name: Set up QEMU
        uses: docker/setup-qemu-action@v3

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3

      - name: Log in to Docker Hub
        uses: docker/login-action@v3
        with:
          username: ${{ secrets.DOCKER_HUB_USERNAME }}
          password: ${{ secrets.DOCKER_HUB_PASSWORD }}

      - name: Log in to Harbor
        uses: docker/login-action@v3
        with:
          registry: harbor.3key.company
          username: ${{ secrets.DOCKER_HUB_USERNAME }}
          password: ${{ secrets.DOCKER_HUB_PASSWORD }}

      - name: Extract metadata (tags, labels) for Docker
        id: meta
        uses: docker/metadata-action@v5
        with:
          images: |
            3keycompany/czertainly-auth
            harbor.3key.company/czertainly/czertainly-auth
          tags: |
            type=ref,event=tag
            type=raw,value=develop-latest
            type=sha,prefix=develop-,format=long

      - name: Build and push Docker image
        uses: docker/build-push-action@v5
        id: build-and-push
        with:
          context: .
          platforms: linux/amd64,linux/arm64
          file: ./Dockerfile
          push: true
          tags: ${{ steps.meta.outputs.tags }}
          labels: ${{ steps.meta.outputs.labels }}

      - name: Sign images with a key
        run: |
          images=""
          for tag in ${TAGS}; do
            images+="${tag}@${DIGEST} "
          done
          cosign sign --yes --key env://COSIGN_PRIVATE_KEY ${images}
        env:
          TAGS: ${{ steps.meta.outputs.tags }}
          COSIGN_PRIVATE_KEY: ${{ secrets.COSIGN_PRIVATE_KEY }}
          COSIGN_PASSWORD: ${{ secrets.COSIGN_PASSWORD }}
          DIGEST: ${{ steps.build-and-push.outputs.digest }}

      - name: Push README to Docker Hub
        uses: christian-korneck/update-container-description-action@v1
        env:
          DOCKER_USER: ${{ secrets.DOCKER_HUB_USERNAME }}
          DOCKER_PASS: ${{ secrets.DOCKER_HUB_PASSWORD }}
        with:
          destination_container_repo: 3keycompany/czertainly-auth
          provider: dockerhub

      - name: Push README to Harbor
        uses: christian-korneck/update-container-description-action@v1
        env:
          DOCKER_USER: ${{ secrets.DOCKER_HUB_USERNAME }}
          DOCKER_PASS: ${{ secrets.DOCKER_HUB_PASSWORD }}
        with:
          destination_container_repo: harbor.3key.company/czertainly/czertainly-auth
          provider: harbor2

Solution

  • Follow the platforms in this build:

    FROM --platform=$BUILDPLATFORM mcr.microsoft.com/dotnet/aspnet:8.0 AS base
    # ...
    FROM --platform=$BUILDPLATFORM mcr.microsoft.com/dotnet/sdk:8.0 AS build
    # ...
    FROM build AS publish
    # ...
    FROM base AS final
    

    The base stage is configured with --platform=$BUILDPLATFORM which means the filesystem used for that image comes from the architecture of the build platform. The same goes for the build stage. Then the publish and final stages use the filesystem of those previous stages which means you always output an image containing binaries of the build platform, regardless of the target platform you assign to the image.

    If you want to build a proper multi-platform image, the resulting image cannot have binaries in it that are compiled for the build platform, they must all be compiled for the target platform.

    You can see this by inspecting the layers of the recently generated image:

    $ regctl manifest get 3keycompany/czertainly-auth:develop-650ce970582573da459921d71c417d272711b94d --platform linux/amd64
    Name:        3keycompany/czertainly-auth:develop-650ce970582573da459921d71c417d272711b94d
    MediaType:   application/vnd.oci.image.manifest.v1+json
    Digest:      sha256:c7583ee357c6f459823b2248f7fc1ae46825258af1eed66fa04c19247dc81345
    Total Size:  97.394MB
    
    Config:
      Digest:    sha256:23a776c53070d10464706037f4f899ae8cbc0bf8016adae5d5b708402afc0c3c
      MediaType: application/vnd.oci.image.config.v1+json
      Size:      5966B
    
    Layers:
    
      Digest:    sha256:2cc3ae149d28a36d28d4eefbae70aaa14a0c9eab588c3790f7979f310b893c44
      MediaType: application/vnd.oci.image.layer.v1.tar+gzip
      Size:      29.150MB
    
      Digest:    sha256:31a19173cb5f4e2626a27effbfa4e008cd5361bb5b441419ca67a1f252440396
      MediaType: application/vnd.oci.image.layer.v1.tar+gzip
      Size:      18.523MB
    
      Digest:    sha256:9af524ca44a59a1c23acd5539176e6d071fb9c81f60112ef057b03b2e818d8a2
      MediaType: application/vnd.oci.image.layer.v1.tar+gzip
      Size:      3282B
    
      Digest:    sha256:62293a3da912ff48dfbcacca57a0f1796844dbbffc87668444695e8e9e5cdda1
      MediaType: application/vnd.oci.image.layer.v1.tar+gzip
      Size:      32.228MB
    
      Digest:    sha256:16d490989d5f6d0f7e2720b6b1ab36c88920659ce2a53abc6353d546b930ee1f
      MediaType: application/vnd.oci.image.layer.v1.tar+gzip
      Size:      154B
    
      Digest:    sha256:4d4fd2f1bd9ccd3f7fe3d567e969e0ab648123d2fd8e153c36f9c8ebdc99763e
      MediaType: application/vnd.oci.image.layer.v1.tar+gzip
      Size:      11.030MB
    
      Digest:    sha256:1eb8437b2d721510db64043c873c7ebb872c4c8198ece10f5241210abff804a5
      MediaType: application/vnd.oci.image.layer.v1.tar+gzip
      Size:      93B
    
      Digest:    sha256:98e9c565ebd97ee3ec97d4f8305b302564497c50cf0db7401980d7eacc9bc5b5
      MediaType: application/vnd.oci.image.layer.v1.tar+gzip
      Size:      1211B
    
      Digest:    sha256:74e6eb212db524678bb921c57addaa4895606dfaf55e069e3fc603a801d309b3
      MediaType: application/vnd.oci.image.layer.v1.tar+gzip
      Size:      6.457MB
    
      Digest:    sha256:b30a27144e8cc840e458e051e7cb2542e61d4484a4522fb7193f0c204de3c46d
      MediaType: application/vnd.oci.image.layer.v1.tar+gzip
      Size:      950B
    
      Digest:    sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1
      MediaType: application/vnd.oci.image.layer.v1.tar+gzip
      Size:      32B
    
    $ regctl manifest get 3keycompany/czertainly-auth:develop-650ce970582573da459921d71c417d272711b94d --platform linux/arm64
    Name:        3keycompany/czertainly-auth:develop-650ce970582573da459921d71c417d272711b94d
    MediaType:   application/vnd.oci.image.manifest.v1+json
    Digest:      sha256:8f31b47fff6b91773f940eb837bdc7f4541d0772306da38df2a9f4d516514131
    Total Size:  97.395MB
    
    Config:
      Digest:    sha256:9d819b278d447c83787eea47f79414363c8cb84b28a0988cacfb6b80bd75c833
      MediaType: application/vnd.oci.image.config.v1+json
      Size:      5965B
    
    Layers:
    
      Digest:    sha256:2cc3ae149d28a36d28d4eefbae70aaa14a0c9eab588c3790f7979f310b893c44
      MediaType: application/vnd.oci.image.layer.v1.tar+gzip
      Size:      29.150MB
    
      Digest:    sha256:31a19173cb5f4e2626a27effbfa4e008cd5361bb5b441419ca67a1f252440396
      MediaType: application/vnd.oci.image.layer.v1.tar+gzip
      Size:      18.523MB
    
      Digest:    sha256:9af524ca44a59a1c23acd5539176e6d071fb9c81f60112ef057b03b2e818d8a2
      MediaType: application/vnd.oci.image.layer.v1.tar+gzip
      Size:      3282B
    
      Digest:    sha256:62293a3da912ff48dfbcacca57a0f1796844dbbffc87668444695e8e9e5cdda1
      MediaType: application/vnd.oci.image.layer.v1.tar+gzip
      Size:      32.228MB
    
      Digest:    sha256:16d490989d5f6d0f7e2720b6b1ab36c88920659ce2a53abc6353d546b930ee1f
      MediaType: application/vnd.oci.image.layer.v1.tar+gzip
      Size:      154B
    
      Digest:    sha256:4d4fd2f1bd9ccd3f7fe3d567e969e0ab648123d2fd8e153c36f9c8ebdc99763e
      MediaType: application/vnd.oci.image.layer.v1.tar+gzip
      Size:      11.030MB
    
      Digest:    sha256:1eb8437b2d721510db64043c873c7ebb872c4c8198ece10f5241210abff804a5
      MediaType: application/vnd.oci.image.layer.v1.tar+gzip
      Size:      93B
    
      Digest:    sha256:98e9c565ebd97ee3ec97d4f8305b302564497c50cf0db7401980d7eacc9bc5b5
      MediaType: application/vnd.oci.image.layer.v1.tar+gzip
      Size:      1211B
    
      Digest:    sha256:f242961bc345d1febacfc14ed433bc12286aa7e6491bfd6a6b53fdda67c92690
      MediaType: application/vnd.oci.image.layer.v1.tar+gzip
      Size:      6.458MB
    
      Digest:    sha256:b30a27144e8cc840e458e051e7cb2542e61d4484a4522fb7193f0c204de3c46d
      MediaType: application/vnd.oci.image.layer.v1.tar+gzip
      Size:      950B
    
      Digest:    sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1
      MediaType: application/vnd.oci.image.layer.v1.tar+gzip
      Size:      32B
    

    You can see the first several layers are identical because they include binaries from the same image, binaries that are platform specific because they are for the build platform.