Why are my unit tests using Moq's Verify() non-deterministic?

Question

We are using Moq 4 for our unit tests, specifically controller unit tests. We are using Moq's Verify() method to make sure an error was logged. The problem is the tests can pass but then fail the very next run.

We are using Serilog for our logger.

The Action method looks like

public IActionResult Index(){
    try{
         var data = _repository.GetData();
         return View(data);
    }catch(Exception e){
        Log.Error(e.Message);
    }
}

So the unit test is using

_mockLogger.Verify(x=> x.Write(LogEventLevel.Error,It.IsAny<string>()));

mockLogger is setup in the test's constructor like

var _mockLogger = new Mock<Serilog.ILogger>();
Log.Logger = _mockLogger.Object;
//...

and the repository is mocked to throw an exception when invoked.

When it fails, we are getting the error message

"Moq.MoqException expected invocation on the Mock at least once but was never peformed x=>x.Write(LogEventLevel.Error,It.IsAny<string>())"

Any Ideas?

Answer 1

It's not entirely possible to see what the problem is from the posted code. And I appreciate how hard it can be to make a MCVE for this. So I'm going to take two guesses.

Guess 1: I suspect the cause of your issue is the use of statics in your code, specifically to do with the logger.

I suspect what's happening is that other tests (not shown in the post) are also modifying/defining how the logger should behave, and since the logger is static, the tests are interfering with each other.

Try redesigning the code so that the instance of the logging functionality is dependency injected into the class under test, using serilog's ILogger interface, store this in a readonly field and use that when you want to log.

Guess 2: Based on the part of the post which says "...setup in the test's constructor" you haven't said (or tagged) which testing framework you're using; but the handful that I've used prefer you to do this kind of thing in attributed methods rather than in the constructor of the test. For example, NUnit has OneTimeSetUp (before any of the tests in that class are run), SetUp (before each test in that class is run), TearDown (after each test in that class is run), OneTimeTearDown (after all of the tests in that class are run). It's possible that the constructors of your tests are being called in an order that you're not expecting, and which is not supported by your testing framework; whereas the attributed methods sequence is guaranteed by the framework.