Search code examples
amazon-web-servicesdockerbuildamazon-ecrmultiarch

buildx build --platform amd64,arm64 --push creates 3 resources in ECR - 2 images and 1 image index, how do I tag them?


Running this command - docker buildx build --platforms linux/amd64,linux/arm64 -t mytag --push .

It creates an image index and 2 images in my ECR.

  1. How do I tag the images and the image index in one command? For now I can only tag the image index.
  2. How can I know for sure that each image is different arch and the image index contains both images?
  3. If I pull the image index from amd64 or arm64 laptop, would he know to pull the right image for my arch? how? I wanted to have multi arch image, is it the image index?

enter image description here


Solution

    1. How do I tag the images and the image index in one command? For now I can only tag the image index.

    There are a variety of tools out there for this. The ones I frequently recommend are Google's crane, RedHat's skopeo, and my own regclient. E.g. with regclient/regctl:

    regctl image copy $image@$(regctl image digest $image:$tag --platform linux/amd64) $image:${tag}-amd64
    regctl image copy $image@$(regctl image digest $image:$tag --platform linux/arm64) $image:${tag}-arm64
    
    1. How can I know for sure that each image is different arch and the image index contains both images?

    You can inspect the manifest, docker now has this built-in:

    docker buildx imagetools inspect $image:$tag
    

    Or with various other tools:

    regctl manifest get $image:$tag
    crane manifest $image:$tag
    
    1. If I pull the image index from amd64 or arm64 laptop, would he know to pull the right image for my arch? how? I wanted to have multi arch image, is it the image index?

    Yes, the image index is how a registry stores a multi-platform image. It contains an array of descriptors to each other other manifests and their platform. The container runtimes (like docker) know how to pick their local platform from that list. E.g. here's what the busybox multi-platform manifest looks like:

    $ regctl manifest get busybox --format '{{jsonPretty . }}'
    {
      "manifests": [
        {
          "digest": "sha256:51de9138b0cc394c813df84f334d638499333cac22edd05d0300b2c9a2dc80dd",
          "mediaType": "application\/vnd.docker.distribution.manifest.v2+json",
          "platform": {
            "architecture": "amd64",
            "os": "linux"
          },
          "size": 528
        },
        {
          "digest": "sha256:40b22bd39e49b9cea975a391de1caad93d63fadc3d23b7dcbf96ec6656064737",
          "mediaType": "application\/vnd.docker.distribution.manifest.v2+json",
          "platform": {
            "architecture": "arm",
            "os": "linux",
            "variant": "v5"
          },
          "size": 528
        },
        {
          "digest": "sha256:d88e0acf1f43052d87d2acb55823f1074a7fd0f7abfe23c07d00e1616cdd15e0",
          "mediaType": "application\/vnd.docker.distribution.manifest.v2+json",
          "platform": {
            "architecture": "arm",
            "os": "linux",
            "variant": "v6"
          },
          "size": 527
        },
        {
          "digest": "sha256:51a9e8ef37a92df3f231b6f96b86c45a622a9078a642b82aa2f45e96c850d92a",
          "mediaType": "application\/vnd.docker.distribution.manifest.v2+json",
          "platform": {
            "architecture": "arm",
            "os": "linux",
            "variant": "v7"
          },
          "size": 528
        },
        {
          "digest": "sha256:9421d4cc473b282bf48888476604444fa3c74f6e3f5cddfa210e773c534cd33d",
          "mediaType": "application\/vnd.docker.distribution.manifest.v2+json",
          "platform": {
            "architecture": "arm64",
            "os": "linux",
            "variant": "v8"
          },
          "size": 528
        },
        {
          "digest": "sha256:da6b2a01ddb011c518d5b07245d781de7c33d6e4cb058d137e9c06ffce4ad7f3",
          "mediaType": "application\/vnd.docker.distribution.manifest.v2+json",
          "platform": {
            "architecture": "386",
            "os": "linux"
          },
          "size": 528
        },
        {
          "digest": "sha256:5fa4e832d41ced43f8a01216c3d6af2ee1607b697cea7c1171d8eb8e2a951388",
          "mediaType": "application\/vnd.docker.distribution.manifest.v2+json",
          "platform": {
            "architecture": "mips64le",
            "os": "linux"
          },
          "size": 528
        },
        {
          "digest": "sha256:b6b4d55059e0cabc8ed615b2a656303f060e16a24b28cb2c18f71e896795e509",
          "mediaType": "application\/vnd.docker.distribution.manifest.v2+json",
          "platform": {
            "architecture": "ppc64le",
            "os": "linux"
          },
          "size": 528
        },
        {
          "digest": "sha256:92f6eed1de620dd42e1fb36e4a04e394d462b180cb48672189a90cb236df35f7",
          "mediaType": "application\/vnd.docker.distribution.manifest.v2+json",
          "platform": {
            "architecture": "riscv64",
            "os": "linux"
          },
          "size": 527
        },
        {
          "digest": "sha256:a748a1de3d1647c5f4c42c4ccc3d55024a40d9663e3d7d1287e319c342df4d9a",
          "mediaType": "application\/vnd.docker.distribution.manifest.v2+json",
          "platform": {
            "architecture": "s390x",
            "os": "linux"
          },
          "size": 528
        }
      ],
      "mediaType": "application\/vnd.docker.distribution.manifest.list.v2+json",
      "schemaVersion": 2
    }
    

    From the comments:

    why does the image index and the other 2 images are all the same size? shouldn't the image index be much much smaller since it contains only the metadata?

    The size in the manifest list is the size of each of the nested manifests, which is also just JSON. You need to look at the size of the filesystem layers within those image manifests to see each image size:

    $ regctl manifest get busybox@sha256:51de9138b0cc394c813df84f334d638499333cac22edd05d0300b2c9a2dc80dd --format '{{jsonPretty . }}'
    {
      "schemaVersion": 2,
      "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
      "config": {
        "mediaType": "application/vnd.docker.container.image.v1+json",
        "size": 1457,
        "digest": "sha256:827365c7baf137228e94bcfc6c47938b4ffde26c68c32bf3d3a7762cd04056a5"
      },
      "layers": [
        {
          "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
          "size": 2587828,
          "digest": "sha256:2123501b93d459033750d3ea725953060ed9bb83bac7c13e46c675be22b69f4a"
        }
      ]
    }
    
    $ regctl manifest get busybox@sha256:40b22bd39e49b9cea975a391de1caad93d63fadc3d23b7dcbf96ec6656064737 --format '{{jsonPretty . }}'
    {
      "schemaVersion": 2,
      "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
      "config": {
        "mediaType": "application/vnd.docker.container.image.v1+json",
        "size": 1470,
        "digest": "sha256:9c8ab461a9c7a50fbb07e348ab2d47386fd201b7898fe7308fd9c657567bbbd8"
      },
      "layers": [
        {
          "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
          "size": 1931376,
          "digest": "sha256:1ddd1b6487ad850ff46f8fb4859302299e7799c47ba4cd9da3c9b27ad96e50a3"
        }
      ]
    }