android unit-testing xamarin visual-studio-2017 nunit

Xamarin Android Nunit tests discoverable in test explorer, but aren't discovered when running tests. (VS2017)

As the title suggests, I'm trying to create some tests for my Android app, which is using Xamarin.Android. I have downloaded the Nunit VS Templates and created a NUnit 3 Test Project (Android), following the steps here.

I can see the default test in the test explorer, but if I try to run the tests I get the following in my output window:

[18/03/2019 11:24:21 Informational] ------ Discover test started ------
[18/03/2019 11:24:25 Warning] No test is available in C:\Project\NUnit.Tests.Droid1\NUnit.Tests.Droid1.csproj C:\Project\Springboard Warehouse\Springboard Warehouse.csproj. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.
[18/03/2019 11:24:25 Informational] ========== Discover test finished: 0 found (0:00:04.1467419) ==========
[18/03/2019 11:24:51 Informational] ------ Discover test started ------
[18/03/2019 11:24:53 Warning] No test is available in C:\Project\NUnit.Tests.Droid1\NUnit.Tests.Droid1.csproj C:\Project\Springboard Warehouse\Springboard Warehouse.csproj. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.
[18/03/2019 11:24:53 Informational] ========== Discover test finished: 0 found (0:00:01.6753664) ==========
[18/03/2019 11:24:53 Informational] ------ Discover test started ------
[18/03/2019 11:24:54 Informational] NUnit Adapter 3.13.0.0: Test discovery starting
[18/03/2019 11:24:54 Warning] Exception NUnit.Engine.NUnitEngineException, Exception thrown discovering tests in C:\Project\NUnit.Tests.Droid1\bin\Debug\NUnit.Tests.Droid1.dll
[18/03/2019 11:24:54 Warning] The NUnit 3 driver cannot support this test assembly. Use a platform specific runner.
[18/03/2019 11:24:54 Warning]    at NUnit.Engine.Drivers.NUnit3FrameworkDriver.Load(String testAssemblyPath, IDictionary`2 settings)
   at NUnit.Engine.Runners.DirectTestRunner.LoadDriver(IFrameworkDriver driver, String testFile, TestPackage subPackage)
   at NUnit.Engine.Runners.DirectTestRunner.LoadPackage()
   at NUnit.Engine.Runners.TestDomainRunner.LoadPackage()
   at NUnit.Engine.Runners.DirectTestRunner.EnsurePackageIsLoaded()
   at NUnit.Engine.Runners.DirectTestRunner.Explore(TestFilter filter)
   at NUnit.Engine.Runners.MasterTestRunner.Explore(TestFilter filter)
   at NUnit.VisualStudio.TestAdapter.NUnit3TestDiscoverer.DiscoverTests(IEnumerable`1 sources, IDiscoveryContext discoveryContext, IMessageLogger messageLogger, ITestCaseDiscoverySink discoverySink) in D:\repos\NUnit\nunit3-vs-adapter\src\NUnitTestAdapter\NUnit3TestDiscoverer.cs:line 96
[18/03/2019 11:24:54 Warning] Innerexception: System.Runtime.Serialization.SerializationException: Type 'NUnit.Framework.Api.FrameworkController' in assembly 'nunit.framework, Version=3.6.1.0, Culture=neutral, PublicKeyToken=2638cd05610744eb' is not marked as serializable.
   at System.AppDomain.CreateInstanceAndUnwrap(String assemblyName, String typeName, Boolean ignoreCase, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes, Evidence securityAttributes)
   at NUnit.Engine.Drivers.NUnit3FrameworkDriver.CreateObject(String typeName, Object[] args)
   at NUnit.Engine.Drivers.NUnit3FrameworkDriver.Load(String testAssemblyPath, IDictionary`2 settings)
[18/03/2019 11:24:54 Informational] NUnit Adapter 3.13.0.0: Test discovery complete
[18/03/2019 11:24:54 Informational] NUnit VS Adapter 2.1.1.0 discovering tests is started
[18/03/2019 11:24:54 Warning] Dependent Assembly Mono.Android, Version=0.0.0.0, Culture=neutral, PublicKeyToken=84e04ff9cfb79065 of C:\Project\NUnit.Tests.Droid1\bin\Debug\NUnit.Tests.Droid1.dll not found. Can be ignored if not a NUnit C:\Project.
[18/03/2019 11:24:54 Warning] Dependent Assembly Mono.Android, Version=0.0.0.0, Culture=neutral, PublicKeyToken=84e04ff9cfb79065 of C:\Project\Springboard Warehouse\bin\Debug\Springboard Warehouse.dll not found. Can be ignored if not a NUnit C:\Project.
[18/03/2019 11:24:54 Informational] NUnit VS Adapter 2.1.1.0 discovering test is finished
[18/03/2019 11:24:54 Warning] No test is available in C:\Project\NUnit.Tests.Droid1\bin\Debug\NUnit.Tests.Droid1.dll C:\Project\Springboard Warehouse\bin\Debug\Springboard Warehouse.dll. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.
[18/03/2019 11:24:54 Informational] ========== Discover test finished: 0 found (0:00:01.6691257) ==========
[18/03/2019 11:25:31 Informational] ------ Run test started ------
[18/03/2019 11:25:32 Informational] NUnit Adapter 3.13.0.0: Test execution started
[18/03/2019 11:25:32 Informational] Running all tests in C:\Project\NUnit.Tests.Droid1\bin\Debug\NUnit.Tests.Droid1.dll
[18/03/2019 11:25:33 Warning] Exception NUnit.Engine.NUnitEngineException, Exception thrown executing tests
[18/03/2019 11:25:33 Warning] The NUnit 3 driver cannot support this test assembly. Use a platform specific runner.
[18/03/2019 11:25:33 Warning]    at NUnit.Engine.Drivers.NUnit3FrameworkDriver.Load(String testAssemblyPath, IDictionary`2 settings)
   at NUnit.Engine.Runners.DirectTestRunner.LoadDriver(IFrameworkDriver driver, String testFile, TestPackage subPackage)
   at NUnit.Engine.Runners.DirectTestRunner.LoadPackage()
   at NUnit.Engine.Runners.TestDomainRunner.LoadPackage()
   at NUnit.Engine.Runners.DirectTestRunner.EnsurePackageIsLoaded()
   at NUnit.Engine.Runners.DirectTestRunner.Explore(TestFilter filter)
   at NUnit.Engine.Runners.MasterTestRunner.Explore(TestFilter filter)
   at NUnit.VisualStudio.TestAdapter.NUnit3TestExecutor.RunAssembly(String assemblyPath, TestFilter filter) in D:\repos\NUnit\nunit3-vs-adapter\src\NUnitTestAdapter\NUnit3TestExecutor.cs:line 310
   at NUnit.VisualStudio.TestAdapter.NUnit3TestExecutor.RunTests(IEnumerable`1 sources, IRunContext runContext, IFrameworkHandle frameworkHandle) in D:\repos\NUnit\nunit3-vs-adapter\src\NUnitTestAdapter\NUnit3TestExecutor.cs:line 103
[18/03/2019 11:25:33 Warning] Innerexception: System.Runtime.Serialization.SerializationException: Type 'NUnit.Framework.Api.FrameworkController' in assembly 'nunit.framework, Version=3.6.1.0, Culture=neutral, PublicKeyToken=2638cd05610744eb' is not marked as serializable.
   at System.AppDomain.CreateInstanceAndUnwrap(String assemblyName, String typeName, Boolean ignoreCase, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes, Evidence securityAttributes)
   at NUnit.Engine.Drivers.NUnit3FrameworkDriver.CreateObject(String typeName, Object[] args)
   at NUnit.Engine.Drivers.NUnit3FrameworkDriver.Load(String testAssemblyPath, IDictionary`2 settings)
[18/03/2019 11:25:33 Informational] NUnit Adapter 3.13.0.0: Test execution complete
[18/03/2019 11:25:33 Informational] NUnit VS Adapter 2.1.1.0 executing tests is started
[18/03/2019 11:25:33 Warning] Dependent Assembly Mono.Android, Version=0.0.0.0, Culture=neutral, PublicKeyToken=84e04ff9cfb79065 of C:\Project\NUnit.Tests.Droid1\bin\Debug\NUnit.Tests.Droid1.dll not found. Can be ignored if not a NUnit C:\Project.
[18/03/2019 11:25:33 Informational] NUnit VS Adapter 2.1.1.0 executing tests is finished
[18/03/2019 11:25:33 Warning] No test is available in C:\Project\NUnit.Tests.Droid1\bin\Debug\NUnit.Tests.Droid1.dll. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.
[18/03/2019 11:25:33 Informational] ========== Run test finished: 0 run (0:00:01.7390314) ==========

I have NUnit v3.11.0, nunit.xamarin v3.6.1, and Nunit3TestAdapter v3.13.0 installed from nuget, as well as the Nunit 3 Test Adapter extension installed. Any ideas what could be causing this problem?

Solution

The NUnit Test Adapter does not support running Android tests in the Visual Studio Test Runner. (I think this is a limitation of the VS test runner, but I'm not 100% sure on that!)

You should instead launch NUnit.Tests.Droid1 as an Android app, either in a simulator or on a device. The app will be a GUI test runner, which will allow you to run your tests.

It'll look something a little like this:

One other thing - you app should be referencing the same version of NUnit as nunit.xamarin - so you really want to be pulling in NUnit 3.6.1 rather than NUnit 3.11. That's planned to change in future!