Using AsyncController to help increase concurrency on a legacy ASP.NET MVC 3 project

Question

We have a website that is struggling with concurrent users right now.

Here is the very high-level background of the project:

• Legacy ASP.NET MVC 3 project (.NET 4)
• Can't do any major rewriting of core code
• Main entry point that takes the longest time to execute is the SubmitSearch action on the Search controller. Average time to respond is 5-10 seconds.

So as the second point outlines, we don't want to spend too much time on this project rewriting large sections. However, we want to attempt to increase concurrent users. We're not looking to change anything else or increase performance since it would require much more work.

What we are seeing is that as more people hit SubmitSearch, the web site in general slows down. That's most likely due to all the IIS threads being locked up executing the search.

We are looking to implement AsyncController and making the SubmitSearch action execute on a normal CLR thread. Here's how we wanted to implement it:

Assume this is the original SubmitSearch method:

/// <summary>
/// Submits a search for execution.
/// </summary>
/// <param name="searchData">The search data</param>
/// <returns></returns>
public virtual ActionResult SubmitSearch(SearchFormModel searchData)
{
    //our search code
}

The quickest way we were hoping to convert to AsyncController is to simply do this:

/// <summary>
/// Submits a search for execution.
/// </summary>
/// <param name="searchData">The search data</param>
/// <returns></returns>
protected virtual ActionResult SubmitSearch(SearchFormModel searchData)
{
    //our search code
}

/// <summary>
/// Asynchronous Search entry point
/// </summary>
/// <param name="searchData"></param>
public void SubmitSearchAsync(SearchFormModel searchData)
{
    AsyncManager.OutstandingOperations.Increment();
    System.Threading.Tasks.Task.Factory.StartNew(() =>
    {
        ActionResult result = SubmitSearch(searchData);
        AsyncManager.Parameters["result"] = result;
        AsyncManager.OutstandingOperations.Decrement();
    });

    return;
}

/// <summary>
/// Called when the asynchronous search has completed
/// </summary>
/// <param name="result"></param>
/// <returns></returns>
public ActionResult SubmitSearchCompleted(ActionResult result)
{
    //Just return the action result
    return result;
}

Of course this didn't work because all through-out the code, we are referencing HttpContext.Current, which we know ends up being null in this approach.

So we were then hoping to do this with SubmitSearchAsync:

/// <summary>
/// Asynchronous Search entry point
/// </summary>
/// <param name="searchData"></param>
public void SubmitSearchAsync(SearchFormModel searchData)
{
    AsyncManager.OutstandingOperations.Increment();
    System.Threading.Tasks.Task.Factory.StartNew(() =>
    {
        ActionResult result = null;
        AsyncManager.Sync(() =>
        {
            result = SubmitSearch(searchData);
        });

        AsyncManager.Parameters["result"] = result;
        AsyncManager.OutstandingOperations.Decrement();
    });

    return;
}

This fixes the issue.

So here's my concern:
Does wrapping the execution of SubmitSearch in the AsyncManager.Sync method defeat the purpose of using this model? In other words, when we are within the AsyncManager.Sync method, are we back on the IIS threads, which puts us back at square one?

Thanks

Answer 1

Does wrapping the execution of SubmitSearch in the AsyncManager.Sync method defeat the purpose of using this model? In other words, when we are within the AsyncManager.Sync method, are we back on the IIS threads, which puts us back at square one?

More or less, yes. But unfortunately, in your case, using Task.Factory.StartNew also defeats the purpose of using an async controller. With the approach you're trying to use, you can't win.

IIS threads, threads started by ThreadPool.QueueUserWorkItem, and Task threads, are all taken from the same thread pool.

In order to gain any benefit from async controllers, you need true async methods. In other words, methods like Stream.ReadAsync or WebRequest.GetResponseAsync. These specially-named methods use I/O completion ports instead of normal threads, which use hardware interrupts and operate on a different thread pool.

I wrote about this a long time ago in my answer here: Using ThreadPool.QueueUserWorkItem in ASP.NET in a high traffic scenario. Tasks and awaiters are pretty sweet, but they don't change the fundamental dynamics of the .NET thread pool.

One thing to note is that there is an option, TaskCreationOptions.LongRunning, that you can specify when creating a Task, which essentially informs the framework that the task will be doing a lot of waiting, and in theory the TPL will attempt to avoid scheduling it in the thread pool. In practice, this probably won't be very practical on a high-traffic site because:

1. The framework doesn't actually guarantee that it won't use the thread pool. That's an implementation detail, and the option is simply a hint that you provide.

2. Even if it does avoid the pool, it still needs to use a thread, which is essentially like using new Thread - if not literally then at least effectively so. What this means is heavy context-switching, which absolutely kills performance and is the main reason why thread pools exist in the first place.

A "search" command clearly implies some kind of I/O, which means there's probably a real asynchronous method you can use somewhere, even if it's the old-style BeginXyz/EndXyz. There are no shortcuts here, no quick fixes; you'll have to re-architect your code to actually be asynchronous.

The .NET framework can't inspect what's going on inside your Task and magically convert it into an interrupt. It simply cannot make use of an I/O completion port unless you refer directly to the specific methods that are aware of them.

Next web or middleware application you work on, try to consider this ahead of time and avoid synchronous I/O like the plague.