HttpClient.GetAsync hangs then internet disappears during its execution

Question

I'm downloading lots of small files (size of each about 1 mb) in a cycle and I want this cycle to break in case of lost internet connection. But httpClient.GetAsync hangs and doesn't throw an exception if I switch my wi-fi off (or take out the cord) while it is being executed.

Also it doesn't get canceled if I'm using CancellationToken (if it is canceled prior to httpClient.GetAsync it gets canceled, but I need to cancel it during the execution).

My experience shows what if internet disappears before httpClient.GetAsync(url, cancellationToken); is called it will throw an exceprion, but if internet disappears then httpClient.GetAsync(url, cancellationToken); already started to execute then it will hang endlessly and even if I trigger set the CancelationTokenSource.Cancel() the operation wouldn't cancel.

The function with HttpClient usage:

private static readonly HttpClient httpClient = new HttpClient();

protected async Task<byte[]> HttpGetData(string url, CancellationToken cancellationToken)
{
    var response = await httpClient.GetAsync(url, cancellationToken);
    response.EnsureSuccessStatusCode();
    return await response.Content.ReadAsByteArrayAsync();
}

It is being called from cycle using line

byte[] data = await LimitedTimeAwaiter<byte[]>.Execute(
async (c) => { return await HttpGetData(chunkUrl, c); }, cancellationToken, 5);

While here is the LimitedTimeAwaiter code

public class LimitedTimeAwaiter<T>
{
    public static async Task<T> Execute(Func<CancellationToken, Task<T>> function, CancellationToken originalToken, int awaitTime)
    {
        originalToken.ThrowIfCancellationRequested();

        CancellationTokenSource timeout = new CancellationTokenSource(TimeSpan.FromSeconds(awaitTime));

        try
        {
            return await function(timeout.Token);
        }
        catch (OperationCanceledException err)
        {
            throw new Exception("LimitedTimeAwaiter ended function ahead of time", err);
        }
    }
}

While I cancel the token given to httpClient.GetAsync doesn't throw an OperationCanceledException and hangs endlessly. I am searching for a way to abort it if doesn't return a value in a limited amount of time.

Answer 1

I'm not 100% sure of what you're trying to do, but there's a flaw in your LimitedTimeAwaiter. You're not actually giving the originalToken to the HttpClient, therefore it won't be used for cancellation. To fix that, you should link your two tokens:

public class LimitedTimeAwaiter<T>
{
    public static async Task<T> Execute(Func<CancellationToken, Task<T>> function, CancellationToken originalToken, int awaitTime)
    {
        originalToken.ThrowIfCancellationRequested();

        var timeout = CancellationTokenSource.CreateLinkedTokenSource(originalToken);
        timeout.CancelAfter(TimeSpan.FromSeconds(awaitTime));

        try
        {
            return await function(timeout.Token);
        }
        catch (OperationCanceledException err)
        {
            throw new Exception("LimitedTimeAwaiter ended function ahead of time", err);
        }
    }
}

(also not entirely sure why you catch the exception but that's beside the point).

Now I'm going to assume your issue is really that the task returned by HttpClient does not complete even when you cancel the right token. First, you should report it because that would be a bug in the HttpClient implementation. Then you can use this workaround:

public class LimitedTimeAwaiter<T>
{
    public static async Task<T> Execute(Func<CancellationToken, Task<T>> function, CancellationToken originalToken, int awaitTime)
    {
        originalToken.ThrowIfCancellationRequested();

        using (var timeout = CancellationTokenSource.CreateLinkedTokenSource(originalToken))
        {
            timeout.CancelAfter(TimeSpan.FromSeconds(awaitTime));

            try
            {
                var httpClientTask = function(timeout.Token);
                var timeoutTask = Task.Delay(Timeout.Infinite, timeout.Token); // This is a trick to link a task to a CancellationToken

                var task = await Task.WhenAny(httpClientTask, timeoutTask);

                // At this point, one of the task completed
                // First, check if we timed out
                timeout.Token.ThrowIfCancellationRequested();

                // If we're still there, it means that the call to HttpClient completed
                return await httpClientTask;
            }
            catch (OperationCanceledException err)
            {
                throw new Exception("LimitedTimeAwaiter ended function ahead of time", err);
            }
        }
    }
}