How to get a file from another repository in GITLAB and use as argument to a python script in current pipeline

I have two projects ProjA and ProjB.

I want to use a yaml file (this is not a gitlab yaml file) from ProjA and pass it to a python script in ProjB's gitlab-ci.yaml file.

Can someone let me know how could we do it ?

Something like:

My ProjectA/common/test.yaml looks like:

test.yaml:

test1:
   a: 1
   b: 2

Now, I have projectB where I want to run a python script as a part of pipeline which parses the above test.yaml

So, my .gitlab-ci looks like:

parse-test:
     python parse_project.py {test.yaml}

What would be easiest way to retrieve test.yaml from ProjA and pass it as argument such that my python script could read this file ?

Solution

You have two solutions : using git clone to get all files or the API to retrieve a specific file.

The $CI_JOB_TOKEN allow you to clone the repository and use files or to make some API calls. This variable is available in your pipeline because it is a predefined variable.

Git Clone

You can clone the projectA during the projectB pipeline to get one or more files.

parse-test:
  stage: test
  script:
    - git clone https://gitlab-ci-token:[email protected]/group/project_a.git
    - cat project_a/common/test.yaml

API

You can use the Repository Files API to get a specific file from projectA :

GET /projects/:id/repository/files/:file_path

Then you have to decode the content and you can save it to a file :

parse-test:
  stage: test
  image: alpine
  before_script:
    - apk add curl jq
  script:
    - test_yaml=$(curl -H "JOB_TOKEN:$CI_JOB_TOKEN" "https://gitlab.com/api/v4/projects/PROJECT_ID/repository/files/common%2Etest%2Eyaml?ref=main")
    - content=$(echo $test_yaml | jq .content | base64 -d)
    - echo "$content" >> test.yaml
    - cat test.yaml