I got an exit code of 2 when when I ran my TestNG test suite. However, according to the output, no tests failed. (Actually, it was run through TeamCity, in a Windows Command-Line step, but I don't think that should make a difference.)
In another run (same test suite on a different System Under Test) I got an exit code of 3. In this case, some tests did fail.
I've googled around and looked at the TestNG documentation, but can't find any information on what TestNG exit codes mean.
What do they mean?
The TestNG exit codes can be summarized as below
/**
* |---------------------|---------|--------|-------------|------------------------------------------|
* | FailedWithinSuccess | Skipped | Failed | Status Code | Remarks |
* |---------------------|---------|--------|-------------|------------------------------------------|
* | 0 | 0 | 0 | 0 | Passed tests |
* | 0 | 0 | 1 | 1 | Failed tests |
* | 0 | 1 | 0 | 2 | Skipped tests |
* | 0 | 1 | 1 | 3 | Skipped/Failed tests |
* | 1 | 0 | 0 | 4 | FailedWithinSuccess tests |
* | 1 | 0 | 1 | 5 | FailedWithinSuccess/Failed tests |
* | 1 | 1 | 0 | 6 | FailedWithinSuccess/Skipped tests |
* | 1 | 1 | 1 | 7 | FailedWithinSuccess/Skipped/Failed tests |
* |---------------------|---------|--------|-------------|------------------------------------------|
*/
There are some issues in the current released version of TestNG viz., 6.12. This has been fixed by me as part of the pull request https://github.com/cbeust/testng/pull/1578
The next upcoming version of TestNG viz., 6.13 onwards, the exit codes will work properly.