Search code examples
selenium-webdrivergitlabautomated-testsgitlab-ciplaywright

Getting an ERROR: Job failed: exit code 1, but I get a warning on the pipeline and a successful/passed email

When a test fails, I get a warning on the pipeline instead of a failure and I get a successful/passed email instead of a failure. This is my .gitlab-ci.yml below

stages:   
  - qa 

image:
    name: maven:3-jdk-11
    pull_policy: [always, if-not-present]

variables:
    MAVEN_OPTS: "-Dmaven.repo.local=/.m2"

test_qa:
  stage: qa  
  script:
    - echo "testing against qa..."
    - mvn test
     
when: manual        
  artifacts:
        when: always
        reports:
            junit:
              - /target/surefire-reports/TEST-*.xml
        expire_in: 120 days

Image of my log, pipeline, and email enter image description here

After the test concludes, I expected the pipeline to fail and the email notification to report the failure. Instead, I am geting a warning and a passed email

Solution

  • If your job shows the warning sign status, it means your job has allow_failure: set (in this case, because it is implied for jobs with when: manual), so the job failure doesn't affect the overall pipeline pass/fail status.

    To resolve this, set allow_failure: false on your job.