Gitlab CI - Split deployment script from build script

Question

Problem description

I have a .gitlab-ci.yaml like this:

stages:
  - build
  - deploy

before_script:
  - docker login ...

build:
  stage: build
  tags:
    - dind
  script:
    - docker pull $CONTAINER_IMAGE:$CI_COMMIT_REF_NAME || true
    - docker build ...
    - docker push $CONTAINER_IMAGE:$CI_COMMIT_REF_NAME
    - docker push $CONTAINER_IMAGE:git-$CI_COMMIT_SHA

manual-deploy-dev:
  image: 'google/cloud-sdk:latest'
  stage: deploy
  tags:
    - dind
  when: manual
  script:
    - echo "$SERVICE_ACCOUNT_DEV_KEY" > key.json
    - gcloud auth activate-service-account --key-file=key.json
    - docker login -u _json_key --password-stdin https://gcr.io < key.json
    - docker pull xxx:git-$CI_COMMIT_SHA
    - docker tag xxx:git-$CI_COMMIT_SHA xxx:git-$CI_COMMIT_SHA
    - docker push xxx

My problem is that since .gitlab-ci.yaml lives in the same branch as the software it implies that if I need to update the deployment script I have to recompile the software for that. And also the hash of the build changes needlessly.

What I want

I'd like to have my code in the master branch and changes should retrigger a build. I would like to have the deployment scripts versioned in the deployment-scripts branch on the same repository.

• branch master - code for the application
• branch deployment-scripts - code for the deployment

And if the deployment-script fails I want to be able to fix it without having to touch the other code.

What I tried

Use before_script

before_script:
      - docker login ...
      - git checkout deployment-scripts

The problem with this solution is that in the .gitlab-ci.yaml there can be only one before_script for all stages and for the build stage I want the 'master' branch and for the 'manual-deploy-dev' I want the deployment-scripts.

This can't work.

Using multi_project_pipelines (with one GL project)

Michael Delgado proposed to use https://docs.gitlab.com/ee/ci/pipelines/multi_project_pipelines.html

But there is a limitation in GL discussed in https://forum.gitlab.com/t/downstream-pipeline-trigger-definition-is-invalid/52356/5 which implies that you need to use two different projects for it to work. In my setup I want to have two different branches.

This can't work with just one GL project.

Using multi_project_pipelines (with two GL projects)

Going with this proposal could easily mean that I would have to have around 6 or 8 more projects for all our different deployment targets in different projects.

This probably works but I'd rather try to avoid that.

Using get raw files from repository with PRIVATE-TOKEN

First I created a read only 'developer' token and for testing hardcoded it into the request. Later I put that token into the environment variables.

curl --header "PRIVATE-TOKEN: xxx" "https://gitlab.example.com/api/v4/projects/567/repository/files/upload_google.sh/raw?ref=deployment-scripts" -o upload_google.sh

Turns out, this is actually the easiest way to do it ATM.

https://docs.gitlab.com/ee/api/repository_files.html#get-raw-file-from-repository

Solution

I use a PRIVATE-TOKEN with the get-raw-file-from-repository method.

Answer 1

I use a PRIVATE-TOKEN with the get-raw-file-from-repository method as outlined in the Solution section of the question.

- 'curl --header "PRIVATE-TOKEN: $READ_ONLY_CI_GOOGLE_UPLOAD_SCRIPT" "https://gitlab.xxx.com/api/v4/projects/$CI_PROJECT_ID/repository/files/upload_google.sh/raw?ref=deployment-scripts" -o upload_google.sh'

Also notice the $CI_PROJECT_ID usage.