C# - HTTP Get run in parallel tasks randomly fails with: Error while copying content to stream

Question

In my C# application (.NET Core 3.1) there's an automatic task that every X hours starts another task, which gets run in parallel multiple times with different parameters. At the end of this automatic task, there's a call to await Task.WhenAll(tasksList). to wait for parallel tasks completion.

Every task issues an HTTPClient (using IHttpClientFactory factory method) and issues a GET request, with the following syntax:

var res = await client.GetAsync(url);
if (res.IsSuccessStatusCode)
{
  var exit = await res.Content.ReadAsStringAsync();
  [...omitted]
}

The issue occurs randomly when two tasks, that share the same GET URL, run at a distance of max 60-70ms. Sometimes both tasks fail, one after another, each with this same exception:

System.Net.Http.HttpRequestException: Error while copying content to a stream.
 ---> System.IO.IOException: The response ended prematurely.
   at System.Net.Http.HttpConnection.FillAsync()
   at System.Net.Http.HttpConnection.ChunkedEncodingReadStream.CopyToAsyncCore(Stream destination, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionResponseContent.SerializeToStreamAsync(Stream stream, TransportContext context, CancellationToken cancellationToken)
   at System.Net.Http.HttpContent.LoadIntoBufferAsyncCore(Task serializeToStreamTask, MemoryStream tempBuffer)

From the logs, I can see how two different HTTP Requests are correctly started and received by the server.

If I remove the ReadAsStringAsync part, the issue never occurs so I presume it's related to the content reading (after the status code check), almost as if the two tasks end sharing the Get result (while having two different active connections issued). I tried using a ReadAsStreamAsync but the issue still occurs (this helps to reduce the occurrence, although).

Another thing that could be related is that the result retrieved is quite heavy (the last time I downloaded it, it ended being a .json file of 4.5MB, more or less).

Should I run each task sequentially? Or am I issuing the HTTP Requests wrong?

IF you want to test this issue, here you can find the source code of a console app I'm using to reproduce the issue (if it doesn't occurs by the first 20calls, restart the app until it occurs):

using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
using System;
using System.Collections.Generic;
using System.IO;
using System.Net.Http;
using System.Text;
using System.Threading.Tasks;

namespace ConsoleApp1
{
    class Program
    {
        static async Task<int> Main(string[] args)
        {
            var builder = new HostBuilder()
                .ConfigureServices((hostContext, services) =>
                {
                    services.AddHttpClient();
                    services.AddTransient<TaskRunner>();
                }).UseConsoleLifetime();

            var host = builder.Build();
            using (var serviceScope = host.Services.CreateScope())
            {
                var services = serviceScope.ServiceProvider;
                try
                {
                    var x = services.GetRequiredService<TaskRunner>();
                    var result = await x.Run();

                    Console.WriteLine(result);
                }
                catch (Exception ex)
                {
                    Console.WriteLine("Error Occured");
                }
            }

            return 0;
        }

        public class TaskRunner
        {
            private static IHttpClientFactory _httpFactory { get; set; }
            public TaskRunner(IHttpClientFactory httpFactory)
            {
                _httpFactory = httpFactory;
            }
            public async Task<string> Run()
            {
                Console.WriteLine("Starting loop...");
                do
                {
                    await Task.Delay(2500); // wait app loading
                    await SendRequest();
                } while (true);
            }
            private static async Task SendRequest()
            {
                await Task.WhenAll(new Task[] { ExecuteCall(), ExecuteCall()};
            }
            private async static Task<bool> ExecuteCall()
            {
                try
                {
                    var client = _httpFactory.CreateClient();
                    // fake heavy API call (> 5MB data)
                    var api = "https://api.npoint.io/5896085b486eed6483ce";
                    Console.WriteLine("Starting call at " + DateTime.Now.ToUniversalTime().ToString("o"));
                    var res = await client.GetAsync(api);
                    if (res.IsSuccessStatusCode)
                    {
                        var exit = await res.Content.ReadAsStringAsync();
                        /* STREAM read alternative
                        var ed = await res.Content.ReadAsStreamAsync();
                        StringBuilder result = new StringBuilder();
                        using var sr = new StreamReader(ed);
                        while (!sr.EndOfStream)
                        {
                            result.Append(await sr.ReadLineAsync());
                        }
                        var exit = result.ToString();
                        */
                        Console.WriteLine(exit.Substring(0, 10));
                        //Console.WriteLine(exit);
                        
                        Console.WriteLine("Ending call at " + DateTime.Now.ToUniversalTime().ToString("o"));
                        return true;
                    }
                    Console.WriteLine(res.StatusCode);
                        Console.WriteLine("Ending call at " + DateTime.Now.ToUniversalTime().ToString("o"));
                    return false;
                }
                catch (Exception ex)
                {
                    // put breakpoint here
                    // Exception => called on line:78 but if content isn't read it never occurs
                    Console.WriteLine(ex.ToString());
                    return false;
                }
            }
        }
    }
}

Thanks for any help/suggestion you can give me!

Answer 1

I'm answering my question to leave the solution I applied, for anyone who can encounter the same issue :)

I added the following line before the Api Call:

var client = _httpFactory.CreateClient();
var api = "https://api.npoint.io/5896085b486eed6483ce";

>>> client.DefaultRequestVersion = HttpVersion.Version10; // new line

var res = await client.GetAsync(api);

The issue seems to be related to the endpoint server, that drops concurrent connections when the HttpVersion is 11. It's possible it relates to the Keep-Alive Connection header, since on 10 v. the header is set to Close.