Search code examples
c#.netunit-testingmicrosoft-fakes

Running multiple async unit test methods produces an error, but running them individually does not


In my WPF MVVM I have two working when ran individually unit tests which test two button command methods:

[TestMethod]
public async Task TestMethod1()
{
    // Arrange
    var interfaceStub = new StubInterface();
    interfaceStub.Method = () => "Message";
    var viewModel = new ViewModel(interfaceStub);

    // Act
    await Task.Run(() => viewModel.GenerateCommand.Execute());

    // Assert
    Assert.AreEqual("Message", viewModel.Response);
}

[TestMethod]
public async Task TestMethod2()
{
    // Arrange
    var interfaceStub = new StubInterface();
    interfaceStub.Method = () => "Message";
    var viewModel = new ViewModel(interfaceStub);

    // Act
    await Task.Run(() => viewModel.VerifyCommand.Execute());

    // Assert
    Assert.AreEqual("Message", viewModel.Response);
}

VerifyCommand and GenerateCommand are actually of type AsyncRelayCommand which implements ICommand, thus they are async void and I have to run them on a separate thread in the test by using Task.Run(). Method1 and Method2 actually stands for the ExecuteMethod1CommandAsync and ExecuteMethod2CommandAsync commands used by buttons respectively.

Problem is, if I run these tests separately, they both pass. However, once I try to test them both at the same time, Assert.AreEqual() fails:

Expected: <"Message"> , Actual <>

This is clearly because one of the tests doesn't wait for the thread to finish and the value is not yet returned at the assertion step. I added the System.Threading.Thread.Sleep(5000); to both test methods and then both tests passed when I ran them together again (which further proves this).

Why does running them individually makes the test methods to wait, but running them separately makes one of them to not wait?

EDIT: When I run them separately, the time it takes: ~140 ms both. When I run them together: the one that passes is ~140 ms again, the other which doesn't is only 90 ms (proves my findings)

EDIT 2: Execute method:

// Executes the action
public async void Execute(object parameter)
{
    // Do something

     try
     {
         await execute(parameter);
     }
     finally
     {
         // Do something
     }
}

execute is a Func<object, Task>. This is apparently where the problem occurs - when execute(parameter) is started, it instantly returns back to the Task in TestMethod. What's the best approach to solve this?


Solution

  • async void means that there's no way for the caller to determine when that activity has completed. Wrapping it in Task.Run() doesn't change that. You now have a Task that starts something running and then its work is done so it's marked as complete despite the fact that the async void method it called may not yet have completed.

    You're not going to fix this without some kind of re-design of the code. If you're constrained by external factors (interface/delegate signatures) to having these methods be void, consider whether whatever is on the other side will be expecting these methods to return when they've not finished. If that's the case, then unfortunately the best way forward is to undo the async stuff you've done there.

    If there are no external factors, make the methods async Task instead so that you can directly await them (no need for Task.Run).

    If there are external factors but you believe the contract allows you to return with work incomplete, then I'd recommend doubling the number of methods you have. Make one set by async Task and unit test these. Then have the async void methods be the thinnest possible wrapper that just calls the async Task version. (If you can accept that these wrappers will not be unit tested).