Automatic removing of danging images and supporting cache at same time for multi-stage builds

I am trying to set up a CI build that builds an image from Dockerfile that contains a multi-step configuration.

Here are the requirements:

It is required to use Docker layers cache because it loads a lot of app dependencies during RUN instruction.
It should automatically remove old dangling images after each build. (To prevent cleaning it manually in future docker image prune)

My sample solution:

(just to demonstrate the issue) looks like this:

docker build -t sample-app.
docker rmi $(docker images --filter "dangling=true" -q --no-trunc)

Dockerfile:

FROM ubuntu:18.04 AS builder
WORKDIR /app
COPY some-file-for-build.txt /app
RUN echo 'building...'

FROM ubuntu:18.04
CMD ["echo", "running..."]

The issue:

Because of removing dangling images the cache becomes invalid and RUN instruction is performed during each build.

Unexpected behavior with single-stage config

But if remove 2nd step, so change Dockerfile to this:

FROM ubuntu:18.04
WORKDIR /app
COPY some-file-for-build.txt /app
RUN echo 'building...'

The cache becomes working for the RUN instruction.

Step 4/4 : RUN echo 'building...'
 ---> Using cache
 ---> 9c7c15230600

Questions:

Is there any way to make it work according to the requirements?
Why does cache works with the single-stage configuration but doesn't work with the multi-stage one?

Solution

You need to be on Docker version 18.09 and above and also need to set an environment variable DOCKER_BUILDKIT=1. This environment variable enables usage of the BuildKit feature, which in turn allows higher performance docker builds and caching possibility to decrease build times. You can get further information here
As far as I have encountered, cache works for both single-stage and multi-stage configuration using Docker >18.09.

Dockerfile:

FROM ubuntu:18.04 AS builder
WORKDIR /app
COPY some-file-for-build.txt /app
RUN echo 'building...'

FROM ubuntu:18.04
WORKDIR /app
COPY --from=builder /app/some-file-for-build.txt some-file-for-build-from-builder-stage.txt
CMD ["bash"]

Here are the logs of my docker build and cleanup:

> docker build --pull --rm -f "61977760.Dockerfile" -t 61977760:latest "."
[+] Building 4.7s (10/10) FINISHED                                                                                                                                       
 => [internal] load .dockerignore                                                                                                                                   0.0s
 => => transferring context: 2B                                                                                                                                     0.0s
 => [internal] load build definition from 61977760.Dockerfile                                                                                                       0.1s
 => => transferring dockerfile: 289B                                                                                                                                0.0s
 => [internal] load metadata for docker.io/library/ubuntu:18.04                                                                                                     2.4s
 => [internal] load build context                                                                                                                                   0.0s
 => => transferring context: 44B                                                                                                                                    0.0s
 => [builder 1/4] FROM docker.io/library/ubuntu:18.04@sha256:3235326357dfb65f1781dbc4df3b834546d8bf914e82cce58e6e6b676e23ce8f                                       0.3s
 => => resolve docker.io/library/ubuntu:18.04@sha256:3235326357dfb65f1781dbc4df3b834546d8bf914e82cce58e6e6b676e23ce8f                                               0.0s
 => => sha256:3235326357dfb65f1781dbc4df3b834546d8bf914e82cce58e6e6b676e23ce8f 1.42kB / 1.42kB                                                                      0.0s
 => => sha256:b58746c8a89938b8c9f5b77de3b8cf1fe78210c696ab03a1442e235eea65d84f 1.15kB / 1.15kB                                                                      0.0s
 => => sha256:c3c304cb4f22ceb8a6fcc29a0cd6d3e4383ba9eb9b5fb552f87de7c0ba99edac 3.41kB / 3.41kB                                                                      0.0s
 => [builder 2/4] WORKDIR /app                                                                                                                                      0.0s
 => [builder 3/4] COPY some-file-for-build.txt /app                                                                                                                 0.0s
 => [builder 4/4] RUN echo 'building...'                                                                                                                            0.8s
 => [stage-1 3/3] COPY --from=builder /app/some-file-for-build.txt some-file-for-build-from-builder-stage.txt                                                       0.0s
 => exporting to image                                                                                                                                              0.9s
 => => exporting layers                                                                                                                                             0.8s
 => => writing image sha256:94fd1e56c8044f0e2f80ee83c784cca5f2c384fd4eba9210f9ff81db68f406e5                                                                        0.0s
 => => naming to docker.io/library/61977760:latest                                                                                                                  0.0s
> docker rmi $(docker images --filter "dangling=true" -q --no-trunc)      
Deleted: sha256:4d157078805ccf08fd42502b19a665a0b60e80a33a7c5b6485e6fd201ca91cc1
Deleted: sha256:6f7f45cca948cf855afea76ff73878b8348da6ed2652d62b48f992f7d9768822
Deleted: sha256:eefe65e58b69e4567c5145828ec95d1d9f3477709cc76fb571e204920df8e378
> docker build --pull --rm -f "61977760.Dockerfile" -t 61977760:latest "."
[+] Building 0.9s (10/10) FINISHED                                                                                                                                       
 => [internal] load .dockerignore                                                                                                                                   0.0s
 => => transferring context: 2B                                                                                                                                     0.0s
 => [internal] load build definition from 61977760.Dockerfile                                                                                                       0.1s
 => => transferring dockerfile: 47B                                                                                                                                 0.0s
 => [internal] load metadata for docker.io/library/ubuntu:18.04                                                                                                     0.8s
 => [builder 1/4] FROM docker.io/library/ubuntu:18.04@sha256:3235326357dfb65f1781dbc4df3b834546d8bf914e82cce58e6e6b676e23ce8f                                       0.0s
 => [internal] load build context                                                                                                                                   0.0s
 => => transferring context: 42B                                                                                                                                    0.0s
 => CACHED [builder 2/4] WORKDIR /app                                                                                                                               0.0s
 => CACHED [builder 3/4] COPY some-file-for-build.txt /app                                                                                                          0.0s
 => CACHED [builder 4/4] RUN echo 'building...'                                                                                                                     0.0s
 => CACHED [stage-1 3/3] COPY --from=builder /app/some-file-for-build.txt some-file-for-build-from-builder-stage.txt                                                0.0s
 => exporting to image                                                                                                                                              0.0s
 => => exporting layers                                                                                                                                             0.0s
 => => writing image sha256:94fd1e56c8044f0e2f80ee83c784cca5f2c384fd4eba9210f9ff81db68f406e5                                                                        0.0s
 => => naming to docker.io/library/61977760:latest                                                                                                                  0.0s
> docker builder prune --all                                              
WARNING! This will remove all build cache. Are you sure you want to continue? [y/N] y
Deleted build cache objects:
np9zknowpkgi9aa2qtfstthnr
l0lucvc40nl98ceqxqadjplaw
elj14uwncgbk928oawwalht9x
s4l0w4h3day53tc3w20njd9cm
t6br3zadzcqhwilswvhkc2qkv
q8wrflxp3sppgmrft0yv805l0
qbe59a3kgjdvaw4ng7d5rasc8
sha256:82751c4dcc36265daceeee0abd938806b591eb66700863186377348c96e8aead
sha256:24b16cac34768092d57cdb429a12de6c9ee870a3724d70098a189ae0ac344c42
sha256:b1daff3b271fede2bec31812c398e28952f792935f608e467271cdfaccab7de2
sha256:b7f7d2967507ba709dbd1dd0426a5b0cdbe1ff936c131f8958c8d0f910eea19e

Total reclaimed space: 64.21MB
>