Say that we have the following unit test:
[Test]
public void OneAndOneTest(int one)
{
Assert.That(1 + one, Is.EqualTo(2));
}
The compiler will accept this code. But the test itself is incorrect as it should have been [TestCase(1)] instead of [test]. When running this test specifically with Nunit, it will give an error saying there is a parameter mismatch. However, when I run all test in a solution, I get a 'skipped' result instead of the said error.
Locally this is an annoyance. But you should have tested your test when writing them, so it's not too big of an issue. (should is the keyword here...)
However, when Jenkins runs said to test it should always fail. Failing the build on skipped tests is not an option as the place I work at allows for skipped tests in some cases.
The problems are with Nunit (I think). Both my local system and Jenkins run the NUnit executable and interpenetrate the output from the executable. My local system ports it into Dot cover while Jenkins has it dumped into an XML file that the Nunit plugin analyses later on in the build process.
Long story short: Is there, with Nunit, a way to fail the build when running all test in a solution with one or more tests having an incorrect amount of parameters
The problem is with NUnit version, I've tried the exact same thing with latest version (NUnit 3.6) and the test result as well as the overall result is "Failed", see output below from NUnit Console:
NUnit Console Runner 3.6.0 Copyright (C) 2017 Charlie Poole Errors, Failures and Warnings
1) Invalid : ConsoleApplication1.Program.add
Method has non-void return value, but no result is expected
Test Run Summary
Overall result: Failed
Test Count: 1, Passed: 0, Failed: 1, Warnings: 0, Inconclusive: 0, Skipped: 0
Failed Tests - Failures: 0, Errors: 0, Invalid: 1