Gitlab CI: How to avoid rebuilding of images on each commit

Question

I'm using Gitlab CI to store and deploy docker images but I got a big issue. Gitlab CI is rebuilding all images on every commit.

The first step is to build my common image which takes around 8 minutes. Currently I only modify child images but the common image is still rebuild on every commit.

Thus the build is a waste of time because the push won't have an effect as the image is already inside the Gitlab repository.

How to avoid the rebuild of images when it's already inside your Gitlab repository?

Below gitlab-ci.yml

build:
   tags:
       - docker_ci_build
   services:
       - docker:dind
   script:
       - docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $CI_REGISTRY
       - >
           docker build 
           -t $CI_PROJECT/common:6.0.1 common

Below a dockerfile:

FROM ubuntu:bionic
RUN apt-get update && \
     DEBIAN_FRONTEND=noninteractive \
     apt-get install -y packages && \
     apt-get clean && \
     rm -rf /var/lib/apt/lists/*
COPY file.conf /etc/common/

Answer 1

Caching appears to be more complicated than what I first thought, and the docker documentation for the various pull options fails to mention any nuances.

If the docker image may or may not yet exist, you can actually adjust the docker pull command to pass your build even if the pull fails:

- docker image pull $LATEST || true

This will pull the image $LATEST if it exists, otherwise will carry on with your build. The next step is to make sure that your build can actually take use of it.

The --pull option for docker build seemed like what you wanted, and docker's documentation says

--pull Always attempt to pull a newer version of the image

but it appears that it doesn't actually do anything for the actual image you're building. More promising is --cache-from which lets you name images that docker can cache from. In my simple test however, I was not seeing docker actually use my cache, probably because it doesn't have enough layers in the docker image to leverage the cache.

- docker image pull $LATEST_IMAGE || true
- docker build --pull -t $TAGGED_IMAGE --cache-from $LATEST_IMAGE .
- docker tag $TAGGED_IMAGE $LATEST_IMAGE
- docker push $LATEST_IMAGE
- docker push $TAGGED_IMAGE

This may be more useful if your build image is multi-stage build, and can actually leverage the cache. Optionally, you could adjust your gitlab-ci script to avoid building if the image exists.

Optionally, you may want to look at gitlab-ci pipelines and creating optional stages and triggers for when you build and push common, and when you build your sub-projects.

I define the variables in a variables section at the beginning of the file Latest is defined with the :latest tag which is basically the most recent build from any CI branch for the project. $TAGGED_IMAGE is defined with :$CI_COMMIT_SHORT_SHA which tags the docker image with the git SHA, which makes kubernetes deployments to a development environment easier to track.