Why docker needs to build all the previous stages?

Question

I need to build 2 stages based on a common one

$ ls
Dockerfile  dev  other  prod

$ cat Dockerfile
FROM scratch as dev
COPY dev /

FROM scratch as other
COPY other /

FROM scratch as prod
COPY --from=dev /env /
COPY prod /

As you can see prod stage does not depend on other stage, however it builds it anyway

$ docker build . --target prod
Sending build context to Docker daemon  4.096kB
Step 1/7 : FROM scratch as dev
 ---> 
Step 2/7 : COPY dev /
 ---> 64c24f1f1d8c
Step 3/7 : FROM scratch as other
 ---> 
Step 4/7 : COPY other /
 ---> 9b0753ec4353
Step 5/7 : FROM scratch as prod
 ---> 
Step 6/7 : COPY --from=dev /dev /
 ---> Using cache
 ---> 64c24f1f1d8c
Step 7/7 : COPY prod /
 ---> 9fe8cc3d3ac1
Successfully built 9fe8cc3d3ac1

Why Docker needs to build the other layers ?

How can I build prod without other ? Do I have to use another Dockerfile ?

Answer 1

There are two different backends for docker build. The "classic" backend works exactly the way you describe: it runs through the entire Dockerfile until it reaches the final stage, so even if a stage is unused it will still be executed. The newer BuildKit backend can do some dependency analysis and determine that a stage is never used and skip over it as you request.

Very current versions of Docker use BuildKit as their default backend. Slightly older versions have BuildKit available, but it isn't the default. You can enable it by running

export DOCKER_BUILDKIT=1

in your shell environment where you run docker build.

(It's often a best practice to run the same Docker image in all environments, and to use separate Dockerfiles for separate components. That avoids any questions around which stages exactly get run.)