Search code examples
c#async-awaithttplistener

Using HttpListener.GetContextAsync() to process requests in parallel

I am trying to get a HttpListener to accept multiple requests in a parallel manner, using an async - await technique. I am aware that in this case "parallel" execution may be limited to "waiting" time, not "processing" time - and this is OK in this case.

The listener itself is managed within a BackgroundService class. At the heart of the logic there is a listening loop as follows (simplified and with a lot of logging removed):

protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
    ...
    while (true)
    {
        var context = await listener.GetContextAsync();
        var processTask = ProcessRequestAsync(context);
    }
    ...
}

The processing function looks like this (simplified and with a lot of logging removed):

private async Task ProcessRequestAsync(HttpListenerContext context)
{
    await Task.Delay(5000);
    context.Response.StatusCode = (int)HttpStatusCode.BadRequest;
    context.Response.Close();
}

As of my current understanding, within the loop the first line should block until there is a request to process, while the second line should not block, instead the processing function in executed until the await is reached, then execution continues, and the loop is repeated with GetContextAsync() being called again.

I would expect that a second request, received within the 5 seconds of waiting by the first request, would be executed immediately (until its own 5 seconds of waiting). If the two requests are made within a second, processing both would thus take ~6 seconds.

What happens instead: the second call to GetContextAsync() occurs timely as expected, but await-ing it blocks until the response of the HttpListenerContext returned by the first call is closed. This causes the two requests to be processed in sequence, taking 10 seconds instead of 6.

This behavior is suggested by another test I did: by moving the await Task.Delay() below Response.Close() , the second await GetContextAsync() no longer blocks, that is it returns immediately as the request is received. This suggests that blocking is not task-related (the duration of the task is identical in the two cases), but rather HttpListenerContext-related.

However, I could not find this behavior documented anywhere...

Has anyone come across this problem? I found a few other similar questions on SO and several solutions suggested - from basic to very sophisticated - but in the end all seem to follow the same principle as the above code - which does not work.

In the past I solved this problem with explicit threading management (and sadly said, it seemed easier) but I am specifically interested whether this can be done with async - await.

A few end notes:

  • Some posters suggested using .configureAwait(false), which I tried but made no difference.
  • I logged the thread used by the processing functions and it is the same. However, I don't know whether this is the cause of the problem or its effect (once the 2 calls are scheduled sequentially, there is no harm in executing both on the same thread).

UPDATES

I did more tests and it keeps getting worse, it looks more and more like a hard limitation of the listener class. To be sure of the results, I splitted the waiting time into halves, 5 seconds before the HTTP response is closed and 5 seconds after that. As a result, two "parallel" request processing operations are overlapped by 5 seconds, meaning that request #2 can only be processed after response #1 was closed.

I got this same result:

  • With the code above, including the suggested improvement using Task.Run()
  • With the older approach to asynchronous processing (BeginGetContext - EndGetContext)
  • With a Thread-based implementation (each context being processed in a different thread)
  • With the code sample included in a related but very old discussion here which was UI-based but I converted it to console app.

All tests were included in very basic console apps, thus the original program being part of a BackgroundService is probably not related to this problem.

As everything seems to revolve around the listener and the listener context (processing of context #2 being processed only after processing of context #1 completes), the problem boils down to whether there is any way to "unlock" context #1 so that context #2 can be processed. I tried - and failed - to achieve this in two naive (and not necessarily usable in production environment) approaches:

  • Closing the HttpListener.Request.InputStream => no changes at all
  • Having the requests served by multiple listeners => failed to start listening on same port (but I suspect this would have gone around the original problem)
Solution

  • I might have found 2 strategy here:

    First solution:

    private async void ThreadBody()
{
   if (!httpServer.IsListening)
   {
      Thread.Sleep(StartDelay);
      httpServer.Prefixes.Add(string.Format("{0}://*:{1}/", EnableHttps ? "https" : "http", port));
      httpServer.Start();
      log?.InfoFormat($"HttpServer<{port}> started");

      while (true)
      {
         HttpListenerContext context = await httpServer.GetContextAsync();
         ThreadPool.QueueUserWorkItem((state) => ProcessRequest(context));
      }
   }
}

    Second solution:

    private async void ThreadBody()
{
   if (!httpServer.IsListening)
   {
      Thread.Sleep(StartDelay);
      httpServer.Prefixes.Add(string.Format("{0}://*:{1}/", EnableHttps ? "https" : "http", port));
      httpServer.Start();
      log?.InfoFormat($"HttpServer<{port}> started");

      while (true)
      {
         HttpListenerContext context = await httpServer.GetContextAsync();
         var reqThProcessor = new Thread(() => ProcessRequest(context));
         reqThProcessor.Start();
      }
   }
}

    Both pretty solid options. I've run tests and with the first solution i got an average of 4.05 requests processed per second and the second solution i got an average of 4.47 requests processed per second, so slightly better. Both start on about 60 req/s but when the threadpool gets filled, in both cases the ratio gets lower.

    In both cases the only time that the server is waiting, is on the GetContextAsync, but as soon as a request is sent, the server opens a new thread for it, and since we are using the async version of GetContext, the server is able to process multiple requests at the same time.

    My application is doing otherstuff also, this could be a reason to the app to get slower on processing requests. But yeah, hope you find this usefull.