Random high number in Custom Performance Counter in Azure Worker Role

Question

I am having a strange issue with Custom Performance counters that I am creating and updating from an Azure Worker Role to keep a track of the number of open connections to my SOAP WCF service.

In the RoleEntryPoint.Run() method I ensure that the counter is created:

if (PerformanceCounterCategory.Exists("WorkerRoleEndpointConnections"))
    return true;

var soapCounter = new CounterCreationData
{
    CounterName = "# SOAP Connections",
    CounterHelp = "Current number of open SOAP connections",
    CounterType = PerformanceCounterType.NumberOfItems32
};
counters.Add(soapCounter);

PerformanceCounterCategory.Create("WorkerRoleEndpointConnections", "Connection statistics for endpoint types", PerformanceCounterCategoryType.MultiInstance, counters);

This appears to work correctly as the counters are correctly created when the Role starts. I have checked via RDP > Perfmon.

Directly after the counter is checked/created I then create a single reference to the counter using the PerformanceCounter object:

m_SoapConnectionCounter = new PerformanceCounter
{
    CategoryName = "WorkerRoleEndpointConnections",
    CounterName = "# SOAP Connections",
    MachineName = ".",
    InstanceName = RoleEnvironment.CurrentRoleInstance.Id,
    ReadOnly = false,
    RawValue = 0L // Initialisation value added in an attempt to fix issue
};

Then I update when required using:

public async Task UpdateSoapConnectionCounter(int connections)
{
    await UpdateCounter(m_SoapConnectionCounter, connections);
}

private async Task UpdateCounter(PerformanceCounter counter, int connections)
{
    await Task.Run(() =>
    {
        counter.RawValue = connections; // Implicit cast to long
    });
}

The idea being that I simply overwrite the value when required in a fire-and-forget manner.

The problem is that this only appears to work sporadically. Randomly the counter will show some large value which is just marginally bigger than int.MaxValue, e.g. 21474836353. Weirdly once this happens it never returns to a "normal" value.

I have tried deleting the counters but even when they are newly created they appear to take on this value (sometimes from the very start), even since adding the initialisation value of Zero when creating the PerformanceCounter object.

I'm at a bit of a loss as to what the problem could be. At first I thought it was just a problem when creating the counter initially, but I have now also observed the counter changing to these values - i.e. 0, 1, 2, 1, 21474836350

I have only found one post which indicates a similar issue, but the only suggestion was to make sure it is initialised because they put the problem down to "uninitialized block of memory that is used to store the performance counter variable" - but I have tried doing this without success.

Note that I don't think this is a perfmon issue as I am seeing this via perfmon and also I have the counters being exported using Azure Diagnostics and both show the value.

Anyone any ideas?

Answer 1

Ok, so after much debugging it turned out that the problem was because there were times when I was assigning a negative number to the counter.

It seems that setting the RawValue property to a negative number causes some kind of bit masking and therefore it actually assigns a value equivalent of int.MaxValue - (negative value).