Reporting shell script test errors in Jenkins pipeline without killing job

Question

In a Jenkins Pipeline, what is the recommended way for reporting shell script errors that make the build unstable (ie, scripts that perform a test, but shouldn't stop remaining tests if failed further on)

Ideally, we wouldn't cancel the build if a 'test' script fails, but report the error at the end. This could be achieved by wrapping 'sh' steps with a catch block and marking the build as unstable. However if we use a single pipeline, all these error logs will be mixed up in a single console log (I realise you can also drill down from the pipeline view but it would require hunting).

Is there a recommended way to report these errors? Ideas include:

• saving errors in an array and printing at end
• capturing output and writing a junit file
• some sort of plugin that has a step to record errors? That would mark the build as unstable if any were reported?

e.g.

def errors = []

try {
    sh "check-doc-style.sh | tee check-doc-style.log"
} catch (e)
    errors << "Doc style check failed : ${e}"
}
step([$class: 'ArtifactArchiver', artifacts: 'check-doc-style.log'])

try {
    sh "sanity-checks.sh | tee sanity-checks.log"
} catch (e)
    errors << "Sanity checks failed : ${e}"
}
step([$class: 'ArtifactArchiver', artifacts: 'sanity-checks.log'])

if (errors.size() > 0) {
    currentBuild.result = 'UNSTABLE'
    for (int i = 0; i < errors.size(); i++) {
        echo errors[i]; 
    }
}

For context: We are migrating from a Jenkins setup where we had a dozen jobs per branch to a single Jenkine pipeline Jenkinsfile to represent the whole thing. A lot of jobs were script tests that either passed (exit code 0) or failed. Error messages were viewed by looking at the console logs.

Answer 1

Exit-Code Handling

Shell

As it's being written Pipeline plugin does not support return of exit code or output from sh step. However there is an open ticket which tracks this feature. Thus try / catch / finally would be the default way to handle such situation.

There are other approaches, but theses are rather cumbersome, ie. :

• ignoring non-zero exit code in bash, ex. set +e
• redirecting output to external file

Maven

sh "${mvnHome}/bin/mvn -B -Dmaven.test.failure.ignore verify"
step([$class: 'JUnitResultArchiver', testResults: '**/target/surefire-reports/TEST-*.xml'])

maven.test.failure.ignore will run all tests even if some of then failed. Then JUnitResultArchiver can set build status to 'UNSTABLE'.

Status Reporting

• In my opinion if you're running tests the most meaningful way to keep track of the results is to analyze them with JUnitResultArchiver.

• However if on other hand you would like to collect warnings and then based on importance change result status, think about using one stage or even creating dedicated one, ie. stage "report", where final information could be clearly visible.

Jenkins 2.0 includes Stage View plug-in which presents Pipeline workflow in a clear way.