Running specific test case not triggering in nunit

The project I'm working on has a test project that contains a large set of system tests that run for about 1 hour each. The project has a single modular test function with about 40 TestCase annotations. These tests are to heavy to do in parallel on a single computer so I'm trying to spread them out over multiple buildnodes (1 test per node).

A trimmed version of that code would be:

[TestCase("4.6", "20180001 V320")]
//more test cases
public void TestAutomaticLists(string version, string customer, string machine = "MachineConfig")
{
  //test
}

To do this I compile the code and run the NunitConsoleRunner to explore for tests:

nunit3-console.exe "Tests\SystemTests\bin\x64\Debug\SystemTests.dll" --explore="tests.txt;format=cases" --noheader --nocolor --where "cat == AST"

This produces a file with all test cases. For example:

SystemTests.Tests.AutomaticListTests.TestAutomaticLists("4.6","20180001 V320")

Within github actions I then parse this output and use a matrix build to spread the tests over the nodes.

I know for sure that the nodes get a valid test case, Nunit also gets all the binaries it needs to run. But 0 tests get executed. The output of the following code

write-output "test to run: $test"
nunit3-console.exe "Tests\SystemTests\bin\x64\Debug\SystemTests.dll" --test="$test"

test to run: SystemTests.Tests.AutomaticListTests.TestAutomaticLists("4.6","20180001 V320")
NUnit Console 3.16.3 (Release)
Copyright (c) 2022 Charlie Poole, Rob Prouse
Thursday, July 6, 2023 2:14:03 PM

Runtime Environment
   OS Version: Microsoft Windows NT 6.2.9200.0
   Runtime: .NET Framework CLR v4.0.30319.42000

Test Files
    Tests\SystemTests\bin\x64\Debug\SystemTests.dll

Test Filters
    Test: SystemTests.Tests.AutomaticListTests.TestAutomaticLists(4.6,20180001 V320)


Run Settings
    DisposeRunners: True
    WorkDirectory: D:\workspaces\vacam\vacam
    ImageRuntimeVersion: 4.0.30319
    ImageTargetFrameworkName: .NETFramework,Version=v4.7.2
    ImageRequiresX86: False
    ImageRequiresDefaultAppDomainAssemblyResolver: False
    TargetRuntimeFramework: net-4.7.2
    NumberOfTestWorkers: 4

Test Run Summary
  Overall result: Passed
  Test Count: 0, Passed: 0, Failed: 0, Warnings: 0, Inconclusive: 0, Skipped: 0
  Start time: 2023-07-06 12:14:03Z
    End time: 2023-07-06 12:14:05Z
    Duration: 1.477 seconds

Results (nunit3) saved as TestResult.xml

I have ensured that the right binaries are in there by calling the --explore option again on the node executing the tests. The test I want to run does show up in the list. I'm guessing I'm calling the --test option incorrectly or something, but I'm not seeing it right now. Any help is appreciated!

Solution

NUnit is being told to run the test named

SystemTests.Tests.AutomaticListTests.TestAutomaticLists(4.6,20180001 V320)

Rather than

SystemTests.Tests.AutomaticListTests.TestAutomaticLists("4.6","20180001 V320")

Since that test is not found, it doesn't run anything.

The argument $test has to be set up with the quotes included and must not be passed to NUnit with any added quotes around it. This is very hard to accomplish from the command-line, which is why the --testlist option exists.

I suggest creating a list of files, as you are now doing, and then splitting that file into as many files as you need. That will avoid any problem with extra double quotes on the command-line.