Unable to start Azure Worker Role, exception code 0xe0434352 & 0xC0000035

Question

When running locally in the emulator the web worker works fine. However whenever I update my web worker running on an Azure VM I get the following exceptions exceptions in the event viewer and the role won't start:

Application: WaWorkerHost.exe
Framework Version: v4.0.30319
Description: The process was terminated due to an unhandled exception.
Exception Info: System.AggregateException
Stack: at System.Threading.Tasks.Task.Wait(Int32, System.Threading.CancellationToken) at System.Threading.Tasks.Task.Wait()
at Foo.PushProcess.WorkerRole.Run()
at Microsoft.WindowsAzure.ServiceRuntime.RoleEnvironment.StartRoleInternal() at Microsoft.WindowsAzure.ServiceRuntime.Implementation.Loader.RoleRuntimeBridge.b__2() at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
at System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
at System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
at System.Threading.ThreadHelper.ThreadStart()

Inner Exception: A task was canceled.

Faulting application name: WaWorkerHost.exe, version: 2.6.1198.712, time stamp: 0x54eba731
Faulting module name: KERNELBASE.dll, version: 6.3.9600.17415, time stamp: 0x54505737
Exception code: 0xe0434352
Fault offset: 0x0000000000008b9c
Faulting process id: 0xfb8
Faulting application start time: 0x01d11e3128981a5d
Faulting application path: E:\base\x64\WaWorkerHost.exe
Faulting module path: D:\Windows\system32\KERNELBASE.dll
Report Id: 30631c5c-8a25-11e5-80c6-000d3a22f3ec
Faulting package full name:
Faulting package-relative application ID:

Session "MA_ETWSESSION_WAD_415df88f8a0447178dbd4c18f1349f0e_Foo.PushProcess_Foo.PushProcess_IN_0" failed to start with the following error: 0xC0000035

This is the relevant code:

public override void Run()
{
    Trace.TraceInformation("Foo.PushProcess is running");

    try
    {
        RunAsync(_cancellationTokenSource.Token).Wait(); // This is where the exceptions point to
    }
    catch (Exception ex)
    {
        Trace.TraceError("[WORKER] Run error: " + ex);
    }
    finally
    {
        _runCompleteEvent.Set();
    }
}

public override bool OnStart()
{
    // Set the maximum number of concurrent connections
    ServicePointManager.DefaultConnectionLimit = 12;

    // For information on handling configuration changes
    // see the MSDN topic at http://go.microsoft.com/fwlink/?LinkId=166357.

    bool result = base.OnStart();

    _storageAccount = CloudStorageAccount.Parse(RoleEnvironment.GetConfigurationSettingValue("StorageConnectionString"));
    var queueClient = _storageAccount.CreateCloudQueueClient();
    _pushQueue = queueClient.GetQueueReference("pushes");
    _pushQueue.CreateIfNotExists();

    CreatePushBroker();

    Trace.TraceInformation("Foo.PushProcess has been started");

    return result;
}

private async Task RunAsync(CancellationToken cancellationToken)
{
    while (!cancellationToken.IsCancellationRequested)
    {
        Trace.TraceInformation("Working");
        CloudQueueMessage message = null;
        try
        {
            message = _pushQueue.GetMessage();
            if (message != null)
            {
                ProcessItem(message);
            }
        }
        catch (Exception ex)
        {
            if (message != null && message.DequeueCount > 5)
                _pushQueue.DeleteMessage(message);

            Trace.TraceError("[WORKER] Retrieval Failure: " + ex);
        }

        await Task.Delay(1000, cancellationToken);
    }
}

Note some code has been omitted, however that is all run after the initialisation and in theory isn't reached by this exception.

I am completely at a loss as to what could cause this issue. Any help would be appreciated - even if only to aid me getting a helpful exception.

UPDATE

I have now reduced my code to the below - it is as simple as a web worker can possibly be - and I am still getting the exceptions. I believe that either the old worker is being cached, or there is an issue in the deployment procedure.

public override void Run()
{
    Trace.TraceInformation("Foo.PushProcess is running");

    try
    {
        RunAsync(_cancellationTokenSource.Token).Wait(); // This is where the exceptions point to
    }
    catch (Exception ex)
    {
        Trace.TraceError("[WORKER] Run error: " + ex);
    }
    finally
    {
        _runCompleteEvent.Set();
    }
}

public override bool OnStart()
{
    // Set the maximum number of concurrent connections
    ServicePointManager.DefaultConnectionLimit = 12;

    // For information on handling configuration changes
    // see the MSDN topic at http://go.microsoft.com/fwlink/?LinkId=166357.

    bool result = base.OnStart();

    return result;
}

private async Task RunAsync(CancellationToken cancellationToken)
{
    while (!cancellationToken.IsCancellationRequested)
    {
        Trace.TraceInformation("Working");

        // code removed for testing - no work is being done.

        await Task.Delay(1000, cancellationToken);
    }
}

Answer 1

To fix this issue I simply deleted the original Cloud VM instance which held the worker role, recreated it and re-published the role. From that point it has worked absolutely fine.

I am still unable to determine what caused the error, and have had no further issues like this with any other worker role. My assumption here was that there was a configuration issue with the VM which could not be amended through code or the Azure portal.