Search code examples
c#.netunit-testingappveyor

AppVeyor ignores explicit assemly name for Unit tests

I'm trying to setup AppVeyor for my project (here) and I can't seem to find a way to only have it run tests from the .NET Core Unit test project.

This is the link for the AppVeyor project: ci.appveyor.com/project/Sergio0694/neuralnetwork-net

I also have a .NET Framework Unit test project that executes some GPU-based tests, so they just fail when run with AppVeyor as they're missing both a CUDA GPU and the required cuDNN files.

This is my configuration:

version: 1.0.{build}
image: Visual Studio 2017
before_build:
- cmd: dotnet restore
build:
  verbosity: minimal
test:
  assemblies:
    only:
    - NeuralNetwork.NET.Unit.dll
  categories:
    except:
    - NetworkTest

That NetworkTest category that's skipped is just a group of tests that are very CPU intensive and require some time to run, so I'm just skipping them now to make the builds finish sooner.

The project builds fine, but AppVeyor keeps running the tests in the other NeuralNetwork.NET.Cuda.Unit.dll assembly as well, which fail as expected and cause the whole build to be marked as failing.

So my questions are:

  • Am I doing something wrong here? Why isn't the assembly constraint respected?
  • Is there another way to have AppVeyor only run the tests from NeuralNetwork.NET.Unit.dll?

EDIT: I've tried to set the "all assemblies excluding" option and all of these combinations to specify the .NET Framework Unit test project to skip:

  • NeuralNetwork.NET.Cuda.Unit.dll
  • **\*.NeuralNetwork.NET.Cuda.Unit.dll
  • **\NeuralNetwork.NET.Cuda.Unit.dll

The tests keep being executed (failing as expected), am I missing something here?

EDIT #2: This is the workaround I'm using for now, I've manually excluded all the test categories in the .NET Framework project and so far this seems to be working (even if it's not so clean to see):

version: 1.0.{build}
image: Visual Studio 2017
configuration: Release
before_build:
- cmd: dotnet restore
build:
  verbosity: minimal
test:
  categories:
    except:
    - CuDnnInceptionLayerTest
    - CuDnnLayersTest
    - GpuExtensionsTest
    - SerializationTest
Solution

  • .NET Core tests detection works differently than detection of classic .NET Framework tests. When searching .NET Core tests, AppVeyor look for .csproj files with certain properties, not for assemblies. Assembly filter does not work for .NET Core tests (Categories filter works though).

    Sorry for confusion. Created this issue to implement behavior similar to assembly name filtering, but based on .csproj file names for .NET Core tests auto-detection.

    For now workaround is to do a "black list", e.g. list all .NET Framework assemblies you do not want to run under assemblies/except. You can use wildcard syntax to make it more compact as described here.