Google Cloud Build does not use cache for some of the RUN executions

Question

I have been using Google Cloud Build with cloudbuild.yaml and a Dockerfile. You can find the files below :

cloudbuild.yaml

steps:
- name: 'gcr.io/cloud-builders/docker'
  entrypoint: 'bash'
  args: ['-c', 'docker pull gcr.io/$PROJECT_ID/github.com/videoo-io/videoo-render:latest || exit 0']
- name: 'gcr.io/cloud-builders/docker'
  args: [
            'build',
            '-t', 'gcr.io/$PROJECT_ID/github.com/videoo-io/videoo-render:latest',
            '--cache-from', 'gcr.io/$PROJECT_ID/github.com/videoo-io/videoo-render:latest',
            '.'
        ]
images: ['gcr.io/$PROJECT_ID/github.com/videoo-io/videoo-render:latest']
timeout: 7200s

Dockerfile :

FROM --platform=amd64 ubuntu:22.10

# Use baseimage-docker's init system.
# CMD ["/sbin/my_init"]

ENV GCSFUSE_REPO gcsfuse-stretch

RUN apt-get update && apt-get install --yes --no-install-recommends \
    ca-certificates \
    curl \
    gnupg \
  && echo "deb http://packages.cloud.google.com/apt $GCSFUSE_REPO main" \
    | tee /etc/apt/sources.list.d/gcsfuse.list \
  && curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add - \
  && apt-get update \
  && apt-get install --yes gcsfuse \
  && apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* 

EXPOSE 80

RUN \
  sed -i 's/# \(.*multiverse$\)/\1/g' /etc/apt/sources.list && \
  apt-get update && \
  apt-get -y upgrade && \
  apt-get install -y build-essential && \
  apt-get install -y gcc && \
  apt-get install -y software-properties-common && \
  apt install -y cmake && \
  apt-get install -y make && \
  apt-get install -y clang && \
  apt-get install -y mesa-common-dev && \
  apt-get install -y git && \
  apt-get install -y xorg-dev && \
  apt-get install -y nasm && \
  apt-get install -y xvfb && \
  apt-get install -y byobu curl git htop man unzip vim wget && \
  rm -rf /var/lib/apt/lists/* 


# Update and upgrade repo
RUN apt-get update -y -q && apt-get upgrade -y -q 

COPY . /app
RUN cd /app
RUN ls -la
# Technicly speaking we must be inside the projects directory now.
# DO NOT FORGET to go back to this directory when working.
# CMD bash premake.sh

# Set environment variables.
ENV HOME /root
ENV WDIR /app

# Define working directory.
WORKDIR /app

ARG CACHEBUST=1

RUN cd /app/lib/glfw && cmake -G "Unix Makefiles" && make && apt-get install libx11-dev
RUN apt-cache policy libxrandr-dev
RUN apt install libxrandr-dev

RUN cd /app/lib/ffmpeg && ./configure && make && make install
RUN cmake . && make

# Define default command.
CMD ["bash"]

When Cloud Build builds through the Dockerfile commands, only some commands are cached. For instance the apt install commands are cached :

Step #1: Step 3/19 : RUN apt-get update && apt-get install --yes --no-install-recommends     ca-certificates     curl     gnupg   && echo "deb http://packages.cloud.google.com/apt $GCSFUSE_REPO main"     | tee /etc/apt/sources.list.d/gcsfuse.list   && curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -   && apt-get update   && apt-get install --yes gcsfuse   && apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
Step #1:  ---> Using cache
Step #1:  ---> 57af7779364e

But the following is not cached :

    Step #1: Step 14/19 : RUN cd /app/lib/glfw && cmake -G "Unix Makefiles" && make && apt-get install libx11-dev
    Step #1:  ---> Running in 167e30a29720
    Step #1: CMake Warning:
    Step #1:   No source or binary directory provided.  Both will be assumed to be the
    Step #1:   same as the current working directory, but note that this warning will
    Step #1:   become a fatal error in future CMake releases.
    Step #1: 
    Step #1: 
    Step #1: -- The C compiler identification is GNU 11.3.0
    Step #1: -- Detecting C compiler ABI info
    Step #1: -- Detecting C compiler ABI info - done
    Step #1: -- Check for working C compiler: /usr/bin/cc - skipped
    Step #1: -- Detecting C compile features
    Step #1: -- Detecting C compile features - done

And the following is not cached as well :

Step #1: Step 17/19 : RUN cd /app/lib/ffmpeg && ./configure && make && make install
Step #1:  ---> Running in cafb9a07e2bc
Step #1: install prefix            /usr/local
Step #1: source path               .
Step #1: C compiler                gcc
Step #1: C library                 glibc
Step #1: ARCH                      x86 (generic)
Step #1: big-endian                no
Step #1: runtime cpu detection     yes
Step #1: standalone assembly       yes
Step #1: x86 assembler             nasm

What is the reason for some of these RUN commands are cached and some are not ?

Answer 1

I'm not sure this is related to cloud build, it looks related to docker layers.

If a layer ultimately generates a different hash from what you have cached.. it will be considered a new layer even if the command looks the same.

And I think following layers are also being rebuilt then.