Search code examples
pythonbashshellgitlab-cigitlab-ci-runner

Getting # collapsed multi-line command error in GitLab CI config file scripts


I'm trying to put a multi-line shell script command in the GitLab CI YAML file as shown below:-

test:
  stage: promote-test-reports
  image: "937583223412.dkr.ecr.us-east-1.amazonaws.com/core-infra/linux:centos7"
  before_script:
     - "yum install unzip -y"
  variables:
    GITLAB_PRIVATE_TOKEN: $GITLAB_TOKEN
  allow_failure: true
  script:
  - >
    #!/bin/bash
    cd $CI_PROJECT_DIR
    echo running
    set -o errexit -o pipefail -o nounset
    API="${CI_API_V4_URL}/projects/${CI_PROJECT_ID}"
    AUTH_HEADER="PRIVATE-TOKEN: ${GITLAB_PRIVATE_TOKEN}"
    CHILD_PIPELINES=$(curl -sS --header "${AUTH_HEADER}" "${API}/pipelines/${CI_PIPELINE_ID}/bridges")
    echo "$CHILD_PIPELINES" | jq . > bridges-$CI_PIPELINE_ID.json
    CHILD_PIPELINES=$(echo $CHILD_PIPELINES | jq ".[].downstream_pipeline.id")  
    echo "$CHILD_PIPELINES" | while read cp  
    do
    # Fetch the IDs of their "build:*" jobs that completed successfully
    JOBS=$(curl -sS --header "${AUTH_HEADER}" "${API}/pipelines/$cp/jobs?scope=success")
    echo "$JOBS" | jq . >> job-$cp.json
    JOBS=$(echo "$JOBS" | jq '.[] | select(([.name] | inside(["test", "coverage"])) and .artifacts_file != null) | .id')
    [[ -z "$JOBS" ]] && echo "No jobs in $cp" && continue
    echo "$JOBS" | while read job
      do
        echo 'DOWNLOADING ARTIFACT: $job'
        curl -sS -L --header '${AUTH_HEADER}' --output artifacts-$job.zip '${API}/jobs/$job/artifacts'
      done
    done  
    if ls artifacts-*.zip >/dev/null
    then
      unzip -o artifacts-\*.zip
    else
      echo "No artifacts"
    fi
  when: always
  rules:
    - if: $CI_PIPELINE_SOURCE == 'web'
  artifacts:
    reports:
      junit: "**/testresult.xml"
      coverage_report:
        coverage_format: cobertura
        path: "**/**/coverage.xml"
  coverage: '/TOTAL\s+\d+\s+\d+\s+(\d+%)/'

Error log:-

Complete!
$ #!/bin/bash cd $CI_PROJECT_DIR echo running set -o errexit -o pipefail -o nounset API="${CI_API_V4_URL}/projects/${CI_PROJECT_ID}" AUTH_HEADER="PRIVATE-TOKEN: ${GITLAB_PRIVATE_TOKEN}" CHILD_PIPELINES=$(curl -sS --header "${AUTH_HEADER}" "${API}/pipelines/${CI_PIPELINE_ID}/bridges") echo "$CHILD_PIPELINES" | jq . > bridges-$CI_PIPELINE_ID.json CHILD_PIPELINES=$(echo $CHILD_PIPELINES | jq ".[].downstream_pipeline.id")   echo "$CHILD_PIPELINES" | while read cp   do # Fetch the IDs of their "build:*" jobs that completed successfully JOBS=$(curl -sS --header "${AUTH_HEADER}" "${API}/pipelines/$cp/jobs?scope=success") echo "$JOBS" | jq . >> job-$cp.json JOBS=$(echo "$JOBS" | jq '.[] | select(([.name] | inside(["test", "coverage"])) and .artifacts_file != null) | .id') [[ -z "$JOBS" ]] && echo "No jobs in $cp" && continue echo "$JOBS" | while read job # collapsed multi-line command
/scripts-36531461-3838222101/step_script: eval: line 177: syntax error near unexpected token `do'

The while loop is throwing error (syntax error near unexpected token `do') for some reason which I'm not able to figure it out i.e.

echo "$JOBS" | while read job do

I tried removing the pipe '|' but no vein. Must be some silly mistake I'm doing. Any idea?


Solution

  • Finally, able to figure it out that the problem was with the API calls.