There was a recent post for VS 17.8 updates touting this behavior but I'm unable to see the 'Benchmarks' tab that is visible in the video.
I'm currently running Visual Studio Community 17.8.358.
Below is the (rather simple) Benchmark I'm running. I'm running it by going to `Debug -> Performance Profiler', selecting some benchmark (e.g. CPU Usage) and having it run. It runs the benchmark in Release which is as expected but it doesn't display the BenchmarkDotNet results.
using BenchmarkDotNet.Configs;
using BenchmarkDotNet.Diagnosers;
using BenchmarkDotNet.Running;
namespace MyBenchmarks;
public class Foo
{
}
public class Foo<T> where T : Bar
{
}
public class Bar
{
}
[MemoryDiagnoser]
public class SpecificActivatorVsNew
{
[Benchmark]
public object ActivatorCreateInstance() => Activator.CreateInstance(typeof(Foo));
[Benchmark]
public object NewOperator() => new Foo();
}
public class Program
{
public static void Main(string[] args)
{
BenchmarkRunner.Run<SpecificActivatorVsNew>();
}
}
There was unfortunately a bit of a miscommunication regarding when this feature was actually ready so it only recently became publicly available. In order to create a diagsession that has the benchmarks tab, you'll need to add a specific BenchmarkDotNet diagnoser to your project that will enable the given profiling tool when your benchmark runs. When this diagnoser is added to your benchmark, you won't need to enable or run the profiling tools separately like you would when profiling a normal project - every time the benchmark runs, even if it's from the command line and outside of Visual Studio entirely, the benchmark will be profiled and a diagsession is created if one of those diagnosers is enabled. The nuget package that contains these diagnosers for the Visual Studio profiling tools can be found here: https://www.nuget.org/packages/Microsoft.VisualStudio.DiagnosticsHub.BenchmarkDotNetDiagnosers and contains a README on how to set up its usage. For the sake of convenience I've copied the contents of the current README here:
This nuget package allows you to add Diagnosers to your BenchmarkDotNet benchmark functions that allow you to profile your benchmarks using the performance profiling tools in Visual Studio.
You can find more information on BenchmarkDotNet here: https://benchmarkdotnet.org/
And you can find information on BenchmarkDotNet Diagnosers here: https://benchmarkdotnet.org/articles/configs/diagnosers.html?q=diagnoser
In order for the diagnosers in this nuget package to work, you must have Visual Studio 17.9 Preview 1 or newer installed on the machine running the benchmarks. You can also use an equivalent version of the Visual Studio Remote tools - more information on the remote tools can be found here: https://learn.microsoft.com/en-us/visualstudio/debugger/remote-debugging
We currently provide the following diagnosers in this nuget package:
Each diagnoser will generate a .diagsession file that can be opened in Visual Studio that will contain performance data related to that diagnoser (the CPUUsageDiagnoser for example provides a diagsession file with CPU data in it, the DatabaseDiagnoser provides a diagsession file with data on database operations, etc.)
To add a diagnoser to your benchmark, you can add the diagnoser name as an attribute to the class that contains the benchmarks you want to generate diagsessions for. For example, the following class:
[CPUUsageDiagnoser]
public class Md5VsSha256
{
private const int N = 10000;
private readonly byte[] data;
private readonly SHA256 sha256 = SHA256.Create();
private readonly MD5 md5 = MD5.Create();
public Md5VsSha256()
{
data = new byte[N];
new Random(42).NextBytes(data);
}
[Benchmark]
public byte[] Sha256() => sha256.ComputeHash(data);
[Benchmark]
public byte[] Md5() => md5.ComputeHash(data);
}
Will create a diagsession file when run that contains CPU usage data on two benchmark function runs (Sha256 and Md5).
When the BenchmarkDotNet run completes there will be a few lines in the output that states where the diagsession was output to. Using the above for example will provide something like this for the output:
// * Diagnostic Output - VSDiagnosticsDiagnoser * Collection result moved to 'BenchmarkDotNet_Md5VsSha256_20231218_123326.diagsession'. Session : {7f38bcc2-c692-4266-aa24-b12bc5325ea4} Stopped Exported diagsession file:
Where the "Exported diagsession file" will include the full path to the diagsession file that was generated
There are unfortunately a few known issues that we're hoping to resolve soon but will currently impede usage of this feature that are not currently reflected in the README:
We're still actively developing this scenario and have more features planned for the near future in addition to fixing bugs so if you have any feedback or suggestions feel free to let us know either here or through https://developercommunity.visualstudio.com/home (or through the "Send Feedback" feature in Visual Studio)