I have a Jenkins job that runs Selenium tests on Browserstack via the Nightwatch.js framework. We have an entire suite of tests that nightwatch runs in separate processes, and we need a way to get the pass/fail value back to Jenkins after all tests have run.
I have been trying to use the Nightwatch hook teardown to run a piece of code at the end of each module with if(this.results.failed) { take action }, but I can't figure out what action I need to take in order to make the failure accessible to the Jenkins job.
My initial thought was to create an environment variable, and set it to false any time a test fails, but I think each process runs in a separate subshell, so the variables created/modified by the test module are not accessible.
My second idea was to create a counter in the globals module, but that value does not increment as expected when referenced by a global after method.
I was able to solve this issue using the Jenkins Log Parser Plugin to fail the test in the event that the console output contains the string "TEST FAILURE".
This was accomplished by creating a rule file that contains a single line:
error /TEST FAILURE/