How to add timeout and retry to url call in C#?

Question

I have a .tgz file that I need to download given a url inside a Testing folder. I am able to download the .tgz file successfully from the url using WebClient.

Below is my code:

private void DownloadTGZFile(string url, string fileToDownload)
{
    using (var client = new WebClient())
    {
        client.DownloadFile(url + fileToDownload, "Testing/configs.tgz");
    }
}

I wanted to see on how can I add a timeout to this call so that if url doesn't respond back within a particular time then it should timeout but it can retry for 3 times and then give up. Also I wanted to see on how can I use HttpClient here instead of WebClient considering it is an older BCL class and not recommended.

Answer 1

To download a file with HttpClient you can do:

// Is better to not initialize a new HttpClient each time you make a request, 
// it could cause socket exhaustion
private static HttpClient _httpClient = new HttpClient()
{
    Timeout = TimeSpan.FromSeconds(5)
};

public async Task<byte[]> GetFile(string fileUrl)
{
    using (var httpResponse = await _httpClient.GetAsync(fileUrl))
    {
        // Throws an exception if response status code isn't 200
        httpResponse.EnsureSuccessStatusCode();
        return await httpResponse.Content.ReadAsByteArrayAsync();
    }
}

For more details about socket exhaustion with HttpClient

As you see, to define a timeout for the Http call you should set a timeout while creating a new HttpClient.

---

To implement a retry policy for the previous code, I would install Polly NuGet package and then:

public async Task<byte[]> GetFile(string fileUrl)
{
    return await Policy
       .Handle<TaskCanceledException>() // The exception thrown by HttpClient when goes in timeout
       .WaitAndRetryAsync(retryCount: 3, sleepDurationProvider: i => TimeSpan.FromMilliseconds(300))
       .ExecuteAsync(async () =>
       {
           using (var httpResponse = await _httpClient.GetAsync(fileUrl))
           {
               // Throws an exception if response status code isn't 200
               httpResponse.EnsureSuccessStatusCode();
               return await httpResponse.Content.ReadAsByteArrayAsync();
           }
       });
}

In this case I defined a retry of 3 times with an interval of 300 milliseconds between each tentative. Also note that I didn't defined the retry for every kind of Exception, because if - for example - you put an invalid URL, retrying is nonsense.

At the end, if you want to save that byte array to a file, you can just do:

File.WriteAllBytes(@"MyPath\file.extension", byteArray);