Fail pipeline when error is logged but pipeline pass

I am looking for a way to fail build pipeline when there are logged any errors. I have found many situations(npm, unit test, and other) when exception/errors/problems occurs causing in pipeline to log and show errors but build pipeline results in SUCCESS(no one feel need to check details because pipeline passed and shows green SUCCESS and that is the problem).

Three tasks('script', 'Bash', 'Powershell') have option "failOnStderr" (set to true makes step failing when anything is written to stderr) which seems to be exact resolution to this problem, but other tasks(like 'vstest', 'cake build' and other) do not have.

Yaml scheme for 'task' does not provide option to fail task when error is logged.

Thank you for your help : ]

Solution

For this issue , there are some scenarios that can cause this. I sort out several situations and workarounds here:

Npm : This means your script "swallows" the exit code and exits normally. you need to add a check to your script that would catch the exit code of your npm run lint and exit with the same exit code, something like:

- script: |
    npm install
    npm run lint # Mapped to `eslint src` in package.json
    if [ $? -ne 0 ]; then
        exit 1
    fi
    npm run slint # `stylelint src` in package.json
    npm run build

Please refer to this case for details.

Test: You can try to add failTaskOnFailedTests: true in the task inputs,for example:

- task: PublishTestResults@2
  inputs:
    testRunner: VSTest
    testResultsFiles: '**/*.trx'
    failTaskOnFailedTests: true

Please refer to this case for details.

Defects in a version of the task itself may also cause this phenomenon. In this case, there is either a circular symlink or just too many files in the original path that the helper that was looking through the files failed. The bug is the helper reported success. This issue ended up by updating the Maven task with a newer version of the task library.

In addition, if you use self-hosted agent to run build pipeline, older versions of the agent may also cause this situation. The solution is to update the agent to the latest version. You can refer to this for details.

Hope this helps.