Search code examples
gitlabgitlab-ci-runner

Gitlab CI/CD changes rule, needs and invalid yaml


I have two jobs in my .gitlab-ci.yml file, similar to this:

job1:
    stage: build
    script:
        - cd docker/some-dir
        - make build
        - make run
    rules:
        - changes: 
            - "docker/some-dir/**/*"
          when: always
        - when: never

job2:    
    stage: build
    needs:
        - job1
    script:
        - cd docker/some-dir2
        - make build
        - make run
  • job1 is builing and pushing docker_image_1
  • job2 is building and pushing docker_image_2
  • docker_image_2 is based on docker_image_1 e.g. the Dockerfile starts with FROM: docker_image_1
  • I want to rebuild docker_image_1 only if something was changed under docker/some-dir
  • this is a monorepo, and I want it that way
  • If I delete the rules section from job1 then there is no error, but of course then it rebuilds docker_image_1 for all push events, and I want to avoid that.

GitLab pipline gives me "yaml error" without any detailed error message. I have already checked the reference https://docs.gitlab.com/ee/ci/yaml/index.html#rules but I cannot figure this out.

Why is it invalid?

What I want to achieve is this:

  • if something changes under docker/some-dir then run the job1 (this includes any CI_PIPELINE_SOURCE with a push operation)
  • OR if the pipeline was started for any other reason without a push event (e.g. started manually, or scheduled etc.) then also run job1
  • otherwise do not run job1
  • and finally, run job2 (even if job1 was not run)

Solution

  • While at the time I didnt know it was the root cause, in my comment I suggested using the needs:optional attribute in job2. From the comments you confirmed this did indeed solve the issue so adding it as an answer for completeness.

    The needs keyword requires that before job2 can run job1 must have run. Job2 in your case has not rules and will always run, however job1 has certain rules that means it doesnt run in all cases. In the pipelines where job1 has not run job2 will try to run but should get an error similar to 'job1' job needs 'job2' job, but it was not added to the pipeline. Sometimes this is not evident from within the UI.

    Using the optional attribute allows you to define that job2 should wait until job1 is finished, however if there is no job1 then job2 can just run anyway. The example from the gitlab docs is

    build-job:
      stage: build
    
    test-job1:
      stage: test
    
    test-job2:
      stage: test
      rules:
        - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
    
    deploy-job:
      stage: deploy
      needs:
        - job: test-job2
          optional: true
        - job: test-job1
      environment: production
    
    review-job:
      stage: deploy
      needs:
        - job: test-job2
          optional: true
      environment: review
    

    for more details you can read https://docs.gitlab.com/ee/ci/yaml/#needsoptional