How do I Run Nunit Tests As a Post-Build Task in Visual Studio?

Question

I have a solution in Visual Studio with a bunch of projects under it. A number of these projects are test projects, containing only nunit tests and code relating to tests. For these projects, I want to set up a post-build event that runs the tests for each such test project. I have researched this but haven't found any questions exactly asking to do what I'm asking- see below. I already have the nunit framework installed on Visual Studio via nuget and I can run nunit tests already from within the VS test explorer.

Notes

• Why am I doing this? Well, we don't have time to set up continuous integration or a build server, yet. I just wanted something quick that would fail the build of the solution if any tests fail. In particular, there is a Wix project in our solution that builds last and this depends on all the other projects. So if post-build in any of them fails, it fails the Wix build. This is what we want: we don't want the Wix to build out an MSI unless all tests pass.
• I have already figured this out, but want to share the knowledge with others (https://stackoverflow.blog/2011/07/01/its-ok-to-ask-and-answer-your-own-questions/) & also see if there's a better way to do this.
• Tried and tested in VS2019, but probably applies to other versions too

Links

This asks how to run mstest as a post-build, rather than nunit: Using Post-Build Event To Execute Unit Tests With MS Test in .NET 2.0+

This asks how to run nunit from within VS, but not as a post-build: How to run NUnit tests in Visual Studio 2017?

Answer 1

The first thing you need to do is install nunit.consolerunner if you don't have it already. Install this via NuGet in VS. Once this is done, you can proceed to writing a post-build script for each test project.

The post build script thankfully doesn't need to vary per project. In fact, it doesn't need to vary at all assuming NuGet puts nunit.consolerunner into the expected place. The script will work in both debug and release configurations because the TargetDir and TargetFileName properties from VS are passed down to the console runner as params. Here is the script, you just need to paste it into the post-build section of build events in the properties pane of the respective project:

cd %HOMEPATH%
cd ".nuget\packages\nunit.consolerunner\3.10.0\tools"
nunit3-console.exe $(TargetDir)$(TargetFileName) --where "cat == Unit"  --work=$(TargetDir)\..

The various parts of this can be explained as follows:

• cd %HOMEPATH% is done because nuget usually installs to your user directory
• cd ".nuget\packages\nunit.consolerunner\3.10.0\tools Assuming .nuget exists in your user directory, this will move into the location of the nuget console runner .exe file
• nunit3-console.exe This is the exe file that runs your tests
• $(TargetDir)$(TargetFileName) This is the full path to the DLL containing your tests, created using VS variables in the properties window
• --where "cat == Unit" This is an optional restriction we used to restrict our tests run to a category of unit tests. You can omit this - it's just to show you how you can restrict to categories if you want to. See the docs (link below) for more. Note: the category "Unit" was our own annotation we added. It's not something out of the box. So if you actually add this restriction without annotating your tests, nothing will run!
• --work=$(TargetDir)\.. This puts the TestResult.xml file in the parent folder of the DLL for the project you're building. For VS2019, at the moment anyway, this directory is the bin folder under the main project folder. We used this because we needed to redirect test output. Otherwise multiple projects in the same solution were trying to write to the same default file and were failing due to concurrency issues.

Links

Nunit console runner docs: https://github.com/nunit/docs/wiki/Console-Command-Line

Update: A Less Annoying Way

The above is pretty annoying, because if any of your tests fail, and you then try to run them from VS's integrated test runner, you can't, because you end up in a vicious cycle: running the tests kicks off a build, which kicks off the tests on the console, which fail, which cause your tests in VS not to run. So in order to debug, you have to go rooting around for the results.XML file. Ugh.

A better way is to do the following:

• Create a new project (I called it TestMaster)
• Make this project depend on all other projects you care about
• Add a post-build script which calls all the nunit tests for all your tests projects

We have five test projects. Let's say they're called TestProject1, TestProject2, ... etc. Then this batch file will serve as the post-build script, if it is put in any sub-directory of the main solution directory. It assumes you have Nuget v3.11.1 installed. If not, just change that for whatever version you do have:

@echo off
rem Debug or release
set CONFIG=%1
set HOMEPATH=%2
cd ../
echo Running unit tests for soln. in root dir %cd%
set currentTest=TestProject1
call :RUN_TESTS_FOR_PROJECT
set currentTest=TestProject2
call :RUN_TESTS_FOR_PROJECT
set currentTest=TestProject3
call :RUN_TESTS_FOR_PROJECT
set currentTest=TestProject4
call :RUN_TESTS_FOR_PROJECT
set currentTest=TestProject5
call :RUN_TESTS_FOR_PROJECT
echo Finished running tests
goto :EOF

:RUN_TESTS_FOR_PROJECT
    cd %currentTest%
    cd bin/%CONFIG%
    echo --- Running tests in directory %cd%
    set targetDir=%cd%
    cd %HOMEPATH%
    cd ".nuget\packages\nunit.consolerunner\3.11.1\tools"
    set targetFileName=%currentTest%.dll
    nunit3-console.exe %targetDir%\%targetFileName% --where "cat == Unit" --work=%targetDir%\..
    if errorlevel 1 (       
        echo Tests Failed for %currentTest%
        echo Exiting test run early
        exit %errorlevel%
    )
    cd %targetDir%
    cd ../../../
    goto :EOF

:EOF    

Let's say you put the above script in a folder called Util. The post-build script of your TestMaster project is then:

cd $(SolutionDir)
cd Util
RunTests $(ConfigurationName) %HOMEPATH%