GitLab CI/CD removes old artifacts from pages public folder

Question

I'm trying to deploy multiple versions of a static website to public of GitLab pages from GitLab CI/CD.

My strategy is to have the tag or branch name in the path such as

• pages.foo.com/project/main/site.html (branch)
• pages.foo.com/project/latest/site.html (tag)
• pages.foo.com/project/v42.1/site.html (tag)

So I defined myself a variable DEPLOY_DIR that is either set to $CI_COMMIT_TAG or $CI_COMMIT_BRANCH using CI/CD's rules in conjunction with if.

The variable itself works fine and the site gets copied to the correct destination.

However, GitLab pages seems to only keep the result of the latest built pipeline. Artefacts resulting from previous pipelines are completely gone, even though the destination folders on the pages "webserver" are completely different from one another.

So, for example:

1. commit and push to branch main => successful pipeline build
2. artefacts are copied to public/main/site.html
3. website pages.foo.com/project/main/site.html works fine
4. tag v42.1 is created => successful pipeline build
5. artefacts are copied to public/v42.1/site.html
6. website pages.foo.com/project/v42.1/site.html works fine
7. website pages.foo.com/project/main/site.html is gone (404)

Here's my .gitlab-ci.yml:

stages:
  - build-pdf
  - deploy-html

default:
  tags:
  - small

build-pdf-doctoolchain:
  stage: build-pdf
  artifacts:
    paths:
      - build/pdf/*.pdf
      - build/html5/*.html
  image: doctoolchain/doctoolchain:v3.2.2
  script:
    - doctoolchain . generatePDF generateHTML

pages:
  stage: deploy-html
  variables:
    DEPLOY_DIR: "default"
  rules:
    - if: $CI_COMMIT_TAG != null
      variables:
        DEPLOY_DIR: $CI_COMMIT_TAG
    - if: $CI_COMMIT_BRANCH != null
      variables:
        DEPLOY_DIR: $CI_COMMIT_BRANCH
  script:
    - export
    - echo "deploy dir $DEPLOY_DIR"
    - mkdir -p "public/$DEPLOY_DIR"
    - cp -v build/html5/*.html "public/$DEPLOY_DIR"
  artifacts:
    name: "$CI_PIPELINE_ID"
    expire_in: never
    paths:
      - public

As you can see, I've already tried to use artifacts.name and artifacts.expire_in, but both don't seem to have any effect.

Answer 1

If you are using managed GitLab (On GitLab.com), then this feature (multiple deployments) is not enabled in the free version (But it is in Premium, Ultimate versions), but if you have self-hosted GitLab then you should enabled this feature by administrator (It's disabled by-default).

Reference:

On self-managed GitLab, by default this feature is not available. To make it available, an administrator can enable the feature flag named pages_multiple_versions_setting. On GitLab.com, this feature is not available. This feature is not ready for production use.

Select the Deploy -> Feature flags on the left side project's menu:

Set the pages_multiple_versions_setting feature flag:

Then you should configure the path prefix within pages.path_prefix scope in your .gitlab-ci.yml file. You can find examples and details on this GitHub page In your case you should extend your pages section with the following part:

pages:
  path_prefix: "$DEPLOY_DIR"

Note:

• This feature is not production ready. You can follow the status of the feature in this ticket: https://gitlab.com/gitlab-org/gitlab/-/issues/422145