Gitlab-CI: conditional allow_failure

Question

I recently started implementing automatic tests for my code, and I noticed that the CI does not catch the warnings of the compiler - the tests are shown as successful even when there are warnings. I have initially added a flag for the compiler to turn the warnings into errors and allow_failure=True, but the problem is that the compiler stops in the first warning->error and does not go through the entire compilation. I then used the trick explained here to write the warnings into a file, and then test if the file is not zero:

    - make 2> >(tee make.warnings)
    - test ! -s make.warnings

After the whole compilation is done, this will give an error if there are warnings written to the file - and using allow_failure=True, this works for the cases where I have no errors/warnings, but also when I have warnings. However, if I have real errors, this will also be shown as a warning in CI, and will not stop the pipeline because of the allow_failure=True.

I could not find a way to allow_failure=True depending on something run in the script (without creating a new stage) or using some condition (i.e., if the file is empty or not). Is there a simple way to do this that I'm missing?

Answer 1

Since there is no condition per se, check out GitLab 13.8 (January 2021):

Control job status using exit codes

You can use the allow_failure keyword to prevent failed jobs from causing an entire pipeline to fail.

Previously, allow_failure only accepted boolean values of true or false but we’ve made improvements in this release.

Now you can use the allow_failure keyword to look for specific script exit codes.

This gives you more flexibility and control over your pipelines, preventing failures based on the exit codes.

test_job_1:
  script:
    - echo "Run a script that results in exit code 1. This job fails."
    - exit 1
  allow_failure:
    exit_codes: 137

test_job_2:
  script:
    - echo "Run a script that results in exit code 137. This job is allowed to fail."
    - exit 137
  allow_failure:
    exit_codes:
      - 137
      - 255

See Documentation and Issue.

If you can use as condition an exit status of a script, that would be enough for allow_failure to be more specific.