Search code examples
gitlabenvironment-variablesgitlab-cidevopscicd

How to manage dotenv report artifacts

Context

I am working with dotenv artifacts to pass variables between GitLab jobs.

A job has been created with a script: directive where 2 environment variables are generated to be passed to the other job. An after_script: directive has been created where 1 environment variables are generated to be passed to the other job. For each of the variables, a dotenv is generated following GitLab documentation: "Multiline values in the .env file are not supported.". PACKAGE_NAME, VERSION and SKIP_JOBS variables work properly, for example in curl. E.g.:

job1:
  stage: job1
  needs: ["..."]
  tags:
    - ...
  only:
    - ...
  script:
    - ...
    - export PACKAGE_NAME=$(werf helm show chart *.tgz | grep '^name:' | awk '{print $2}')
    - echo $PACKAGE_NAME
    - echo "PACKAGE_NAME=${PACKAGE_NAME}" >package.env
    - export VERSION=$(werf helm show chart *.tgz | grep '^version:' | awk '{print $2}')
    - echo "VERSION=${VERSION}" >version.env
    - 'curl --request POST --user gitlab-ci-token:$GITLAB_TOKEN --form "chart=@${PACKAGE_NAME}-${VERSION}.tgz" "${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/helm/api/stable/charts"'
  after_script:
    - ...
    - export SKIP_JOB=$(yq ".[].metadata | select(.name == \"$ARGOCD_APP_NAME\")" $HELM_APP_ARGOCD)
    - |- 
        if [[ $SKIP_JOB != '' ]]; then
            echo "SKIP_JOB=True" >dotenv.env
        else
            echo "SKIP_JOB=False" >dotenv.env
        fi
  artifacts:
    reports:
      dotenv: dotenv.env
      dotenv: package.env
      dotenv: version.env
    paths:
      - dotenv.env
      - package.env
      - version.env

The files generated by adding in the artifact section can be observed where each one of them meets the requirement of: "The maximum size of the .env file is 5 KB.". The files contain the environment variables.

enter image description here

In the second job, the 3 variables generated in paper 1 are used. E.g.:

job2:
  stage: job2
  tags:
    - ...
  only:
    - ...
  script:
    - |- 
        if [ "$SKIP_JOB" == "True" ]; then
          echo "This job is skipped based on the condition"
          exit 0
        else
          echo "This job is running"
        fi
    - ...
    - echo $VERSION
    - echo $PACKAGE_NAME
    - ...
  needs:
    - job: job1

Issue

When executing the second job the VERSION variable can be used properly, while PACKAGE_NAME cannot. E.g::

$ echo ${PACKAGE_NAME}
$ echo ${VERSION}
0.0.8

For reference, the variables are obtained from a Chart.yaml file:

apiVersion: v2
name: nameexample
description: ...
type: application
version: 0.0.8
appVersion: vx.x.x

dependencies:
  - ...
    ...

where:

  • PACKAGE_NAME is associated with the chart name.
  • VERSION is associated with the chart version.

Question

Why is it not possible to pass the variable PACKAGE_NAME from job1 to job2?

Solution

  • The problem has been solved by avoiding overwriting variables on a single dotenv file. For which the following operators are used in the variables:

    1. > (Redirect Output)
    2. >> (Append Output)
         - ...
     - export PACKAGE_NAME=$(werf helm show chart *.tgz | grep '^name:' | awk '{print $2}')
     - echo $PACKAGE_NAME
     - echo "PACKAGE_NAME=${PACKAGE_NAME}" >build.env
     - export VERSION=$(werf helm show chart *.tgz | grep '^version:' | awk '{print $2}')
     - echo "VERSION=${VERSION}" >>build.env
     - 'curl --request POST --user gitlab-ci-token:$GITLAB_TOKEN --form "chart=@${PACKAGE_NAME}-${VERSION}.tgz" "${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/helm/api/stable/charts"'
  artifacts:
    paths:
      - build.env
      - ...
    reports:
      dotenv: build.env
      dotenv: ...