Search code examples
cachinggitlabgitlab-ci

GitLab Pages CI cache appears to cache and restore nothing. How do I inspect the cache?


I have a .gitlab-ci.yml that looks like this:

image: mcr.microsoft.com/playwright:latest

before_script:
- apt-get remove nodejs -y
- apt-get update
- apt-get install -y curl
- curl -fsSL https://deb.nodesource.com/setup_18.x | bash -
- apt-get install -y nodejs
- npm install

pages:
  stage: build
  script:
  - npm run docsify ; npx next build ; npx next export
  - rm -rf public
  - mv out public
  artifacts:
    paths:
    - public
  cache:
    key: default
    paths:
    - node_modules/
    - ".next/cache/"
    - public
    untracked: true # unclear if needed
  only:
  - main

And it runs and it looks like it does all the right things:

Running with gitlab-runner 14.9.1 (bd40e3da)
  on runner-gitlab-runner-7ff6fc5d7b-dc9sh 2bfx5V6B
Preparing the "kubernetes" executor
Using Kubernetes namespace: gitlab-managed-apps
Using Kubernetes executor with image mcr.microsoft.com/playwright:latest ...
Using attach strategy to execute scripts...
Preparing environment
Waiting for pod gitlab-managed-apps/runner-2bfx5v6b-project-3385-concurrent-06khk6 to be running, status is Pending
Waiting for pod gitlab-managed-apps/runner-2bfx5v6b-project-3385-concurrent-06khk6 to be running, status is Pending
    ContainersNotInitialized: "containers with incomplete status: [init-permissions]"
    ContainersNotReady: "containers with unready status: [build helper]"
    ContainersNotReady: "containers with unready status: [build helper]"
Running on runner-2bfx5v6b-project-3385-concurrent-06khk6 via runner-gitlab-runner-7ff6fc5d7b-dc9sh...
Getting source from Git repository
Fetching changes with git depth set to 50...
Initialized empty Git repository in 
Created fresh repository.
Checking out 456ffc27 as main...
Skipping Git submodules setup
Restoring cache
Checking cache for default-4...
No URL provided, cache will not be downloaded from shared cache server. Instead a local version of cache will be extracted. 
Successfully extracted cache
Executing "step_script" stage of the job script

...

Saving cache for successful job
Creating cache default-4...
node_modules/: found 22078 matching files and directories 
.next/cache/: found 18 matching files and directories 
public: found 925 matching files and directories   
untracked: found 21814 files                       
No URL provided, cache will be not uploaded to shared cache server. Cache will be stored only locally. 
Created cache

So it should be caching:

  • node_modules/
  • .next/cache/
  • public

My understanding is that after the next git clone of the next pipeline run, GitLab CI should restore the cache on top of the cloned repository. (Or make the cache available via a volume mount, but I haven't been able to find it in /cache or ./cache.)

Printing out the contents of public in the next commit reveals this is not the case.

But by all appearances the cache gets restored:

Fetching changes with git depth set to 50...
Initialized empty Git repository in 
Created fresh repository.
Checking out d7d84cd8 as main...
Skipping Git submodules setup
Restoring cache
Checking cache for default-4...
No URL provided, cache will not be downloaded from shared cache server. Instead a local version of cache will be extracted. 
Successfully extracted cache

Where is the cache supposed to be?

How am I supposed to inspect it to see if it's working?


Addendum:

From a Next.js standpoint, this message would indicate that Next.js is not benefiting from the cache either:

warn  - No build cache found. Please configure build caching for faster rebuilds. Read more: https://nextjs.org/docs/messages/no-cache

And I have the lines that they suggest you merge into your .gitlab-ci.yml to take advantage of caching.


Solution

  • If you want to use caching, you have to set up distributed caching.

    If you want to set up distributed caching, you have to setup an S3-compatible object storage.

    There is no out-of-the-box support for caching without object storage.

    Even if you use the same runner for every build, it does not persist.