NUnit 2.6.3 - Tests not executing with message Test adapter sent back a result for an unknown test case

I am playing with NUnit 2.6.3 and I did this tests:

using NUnit.Framework;
using System;

namespace NUnit26Tests
{
    [TestFixture]
    public class RandomTests
    {
        [Test]
        public void RandomTest([Random(1, 100, 5)] int value)
        {
            Assert.IsTrue(true);
        }

        [Test]
        public void SuccessTests()
        {
            Assert.That(true, Is.True);
        }
    }
}

But most of the execution times (99%) RandomTest is not executing on Test Runner.

This is the output message window:

------ Discover test started ------
NUnit 1.0.0.0 discovering tests is started
NUnit 1.0.0.0 discovering test is finished
========== Discover test finished: 6 found (0:00:00,9970583) ==========
------ Run test started ------
NUnit 1.0.0.0 executing tests is started
Run started: C:\TestProjects\NUnit26Tests\NUnit26Tests\bin\Debug\NUnit26Tests.dll
NUnit 1.0.0.0 executing tests is finished
Test adapter sent back a result for an unknown test case. Ignoring result for 'RandomTest(92)'.
Test adapter sent back a result for an unknown test case. Ignoring result for 'RandomTest(38)'.
Test adapter sent back a result for an unknown test case. Ignoring result for 'RandomTest(69)'.
Test adapter sent back a result for an unknown test case. Ignoring result for 'RandomTest(96)'.
========== Run test finished: 2 run (0:00:09,271531) ==========

In this case only one of five RandomTest's was executed.

I have tested with runner Nuget Package and installing NUnit Runner extension, same result.

Any idea what is the problem ?

Solution

I was able to reproduce this behavior. This seems to be a bug within the NUnit framework and/or the Test Adapter.

My guess is, that the random values are drawn once before the tests are run (to display them) and once when run. The random values drawn will probably not match and so the test results may not be assigned, leading to the mentioned error message.

You could open a bug for this issue at the project's development site (https://launchpad.net/nunitv2), but they are heavily busy with the upcoming v3 release.

As a workaround for your issue I propose that you use static (random) values (not using the RandomAttribute) or draw random values within your test (not as a parameter):

[Test]
[TestCase(15)]
[TestCase(38)]
[TestCase(2)]
[TestCase(72)]
[TestCase(69)]
public void RandomTest(int value)
{
    Assert.IsTrue(true);
}

Update

There is a known issue for this on github.