BenchmarkDotNet, skip a benchmark on specific runtime

Question

(https://benchmarkdotnet.org/)

Is it possible to skip a single benchmark section for a specific runtime?

For example i want to test several functions for 4.7.2 and Core 3.1, but one should only be benched on Core31

[Benchmark]
public void CoreOnly()
{
#if NETCOREAPP3_1
    //some stuff i only want to test with 3.1
#endif
}

[Benchmark]
public void General()
{
    //some stuff i want to test on all runtimes
}

Thats how i have done this until now. Is there a better way?

Answer 1

This is impossible by design.

When running the host process targetting XYZ framework, BDN is using the reflection to obtain the list of available methods (benchmarks). If you are using #if defines, then the list of benchmarks is going to be different per host process target framework.

Performance repo docs describe how to compare multiple runtimes performance here: https://github.com/dotnet/performance/blob/master/docs/benchmarkdotnet.md#multiple-runtimes

The host process needs to be the lowest common API denominator of the runtimes you want to compare!

You can read more about testing multiple runtimes here: https://benchmarkdotnet.org/articles/configs/toolchains.html#multiple-frameworks-support