How is Azure Diagnostics storage account configured?

Question

Reading MSDN I got an impression that the account used for storing Azure Diagnostics is configured like this:

<ConfigurationSettings>
   <Setting name="Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString"
       value="DefaultEndpointsProtocol=https;AccountName=AccountName;AccountKey=AccountKey"/>
</ConfigurationSettings>

(the settings name is the fixed "Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString" string). But then I also found code snippets for setting up the DiagnosticMonitor (from this question by user Søren Randrup:

<ConfigurationSettings>
     <Setting name="DiagnosticsConnectionString"
        value="DefaultEndpointsProtocol=https;AccountName=AccountName;AccountKey=AccountKey" />
</ConfigurationSettings>

var dm = DiagnosticMonitor.GetDefaultInitialConfiguration();
dm.Logs.ScheduledTransferPeriod = TimeSpan.FromSeconds(5);
dm.Logs.ScheduledTransferLogLevelFilter = LogLevel.Verbose;
DiagnosticMonitor.Start("DiagnosticsConnectionString", dm);

(the setting name is a string chosen by the developer).

This looks confusing - I'm under impression that it's the same setting but for whatever reason different people configure it differently.

Where is the storage account specified and why are the two abovementioned snippets seemingly used for the same task?

Answer 1

When DiagnosticsMonitor was first released (1.1 or 1.2 SDK, IIRC), it was started from the RoleEntryPoint (user code). This had some disadvantages, namely, if your RoleEntryPoint crashed, it took the DM down with it. When the 1.3 SDK was released, it changed the design and started the DM as a background startup task. This allows the DM to run in a separate process than your code and hence it no longer is susceptible to being crashed by user code. The "Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString" is the setting defined in the Diagnostics role plugin (look in your bin/plugins directory).

If you do nothing but import the Diagnostics plugin and set this connection string, you will have diagnostics enabled for some default things, but nothing will actually transfer to your storage account for analysis.

So, a more complete solution is to:

1. Enable the DM plugin and set a valid connection string to a storage account in Windows Azure.
2. Either remotely (using tools like scripts, ManageAxis, Windows Azure MMC, Cerebrata, or others) set the configuration for monitoring like you want*
3. Or, at runtime, in your RoleEntryPoint, imperatively configure the monitor via code.

This gets your counters, logs, whatever, actually transferring into your storage account. At this point, it is up to you to interpret it. There are products (e.g. ManageAxis and others) that do the analysis for you (graph, autoscale, etc.), but you are largely on your own to query the data produced by the DM otherwise.

* I prefer this method as I don't like coding this in my RoleEntryPoint and I have tooling that can watch and maintain it.