MVC3/Azure - What's the most scalable way to handle 50k+/second quick API calls?

Question

I have a farm of Web Roles running on Azure based on MVC3 that potentially need to handle ~50k API calls per second. The calls will be quick, all they do is adding some data to Azure Storage Queues (I am aware of the quotas there, I do have my queues partitioned). No values/status are returned back to the client by the API. At the moment each call takes around 100-200ms.

Currently I have it setup as an AsyncController, my code is below.

My question to you all - is this the most scalable, efficient and quickest way to handle this type of load?

PS - I have been told that TaskCreationOptions.LongRunning is recommended, true?

Any help is greatly appreciated!

public class ApiController : AsyncController {
    public void CallAsync(string data) {
        AsyncManager.OutstandingOperations.Increment(1);

        Task.Factory.StartNew(() => {
           // add to my queue here
           AsyncManager.OutstandingOperations.Decrement();
        }, TaskCreationOptions.LongRunning);
    }
    public void CallCompleted() {}
}

Answer 1

No values/status are returned back to the client by the API.

This is almost always a mistake. You seriously don't want your clients notified if the value wasn't saved for some reason? Like, say, an expired Azure Storage certificate? Are you sure that you want your error handling behavior to be "just delete the data and pretend the call never happened"?

For the remainder of this question, I'm going to assume that you do actually want a "fire and forget" API call that will always return success even when it fails, since that's what you specify in your question.

The calls will be quick, all they do is adding some data to Azure Storage Queues.

There are two different ways you can optimize this: for the average case and for the worst case.

If you want to optimize for the average case, then consider using blocking calls. (gasp!)

The average latency for a properly-administrated Azure Storage Queue is a miniscule 10ms, with a transaction time as low as 0.5ms. With speeds that fast, you're probably better off just blocking the ASP.NET request thread rather than switching to a thread pool thread (or starting up a new thread). Plus, your code becomes quite easy to read:

public class ApiController : Controller {
  public void CallAsync(string data) {
    // add to my queue here
  }
}

The disadvantage to the blocking method is that if the Queue latency suddenly goes up, your ASP.NET servers will very quickly fill up with blocked threads and you'll start 503'ing.

If you want to optimize for the worst case, then you should toss the queueing onto a background thread and then (synchronously) return. You can use the BackgroundTaskManager from my blog to register the background tasks with ASP.NET:

public class ApiController : Controller {
  public void CallAsync(string data) {
    BackgroundTaskManager.Run(async () => {
      // add to my queue here
    });
  }
}

The advantage of this is that your servers can "buffer" more in memory as background tasks than as threads. So if the Queue latency spikes and then recovers, your web servers will be able to handle it. The disadvantage of this approach is that the requests may all be slightly slower than the synchronous version.

PS - I have been told that TaskCreationOptions.LongRunning is recommended, true?

Absolutely not in the way you're using it. Your current code is actually creating a new thread for each incoming request - very inefficient. LongRunning should only be used when creating tasks that are long-running.