I have a solution, which contains numerous projects. Each of these projects has an associated test project which uses MSTest V2. Some of these test projects take advantage of Microsoft Fakes.
When I run the tests in the Visual Studio IDE, everything works fine. However, my build pipeline fails. I attempted to emulate what my build pipeline is doing and it also fails on my machine.
The error message I get is the following:
System.TypeLoadException: Could not load type 'System.Fakes.ShimDateTime' from assembly 'mscorlib.4.0.0.0.Fakes, Version=4.0.0.0, Culture=neutral, PublicKeyToken=0ae41878053f6703'.
After playing around with the vstest.console.exe application, it seems that if I run each test assembly separately all of the tests pass, but if I attempt to pass in multiple assemblies which utilize Microsoft Fakes they fail.
vstest.console.exe "Path/To/First.dll" Passesvstest.console.exe "Path/To/Second.dll" Passesvstest.console.exe "Path/To/First.dll" "Path/To/Second.dll" Failsvstest.console.exe "Path/To/First.dll" "Path/To/Second.dll" /InIsolation FailsI'm not sure what is going on here or how to resolve this issue.
I eventually figured out what the problem was:
Test Project A had not defined System.DateTime in a Microsoft Fakes config file, so an assembly was generated named mscorlib.4.0.0.0.Fakes.dll which was missing the System.Fakes.ShimDateTime.
Test Project B had defined System.DateTime in a Microsoft Fakes config file, so an assembly was generated named mscorlib.4.0.0.0.Fakes.dll which contained the type System.Fakes.ShimDateTime.
This lead the following structure:
ProjectA.Tests.dll -- Contains the unit tests for Project Amscorlib.4.0.0.0.Fakes.dll -- Does not contain System.Fakes.ShimDateTimeProjectB.Tests.dll -- Contains the unit tests for Project Bmscorlib.4.0.0.0.Fakes.dll -- Does contain System.Fakes.ShimDateTimeI'm not sure what the Visual Studio IDE is doing differently, but the vstest.console.exe seems to load all of the assemblies, ignoring duplicates based on fully qualified names. That is, it loads the following files:
It ignored the file ProjectB.Tests/bin/Debug/mcsorlib.4.0.0.0.Fakes.dll, which had the type System.Fakes.ShimDateTime defined within it.
The only solution to this is to run a separate vstest.console.exe per test project (instead of all at once), or to ensure that the generated Microsoft Fakes assemblies (*.Fakes.dll) have the same types defined across the board, so it doesn't matter which version of mscorlib.4.0.0.0.Fakes.dll is loaded. And, since the Azure Pipelines VsTest task uses a single run of vstest.console.exe this is important to know.