Multiple web requests, start another request while waiting for response on last one. C#

Question

I am trying to do multiple web requests from websites that take a long time to load the full response (really long log files). I want to implement a method that will continue to create web requests while waiting for the response of the other ones and speed the process.

Tried to implement an async- await method but I don't think is creating simultaneous calls and it's instead running synchronously. I am very confused in how to implement the async await, I looked at other similar questions but still not very clear, how could I fix this code to run async?
This is my code:

        public static async Task<String> MakeRequestAsync(String url)
    {
        String responseText = null;

            try
            {
                HttpWebRequest request = WebRequest.Create(url) as HttpWebRequest;
                request.Proxy = null;
                WebResponse response = request.GetResponse();
                Stream responseStream = response.GetResponseStream();
                string sr = new StreamReader(responseStream).ReadToEnd();
                response.Close();
                responseStream.Close();
                responseText = sr;
            }

            catch (Exception e)
            {
                Console.WriteLine("Error: " + e.Message);
            }
        return responseText;
    }

..

private void button_test_Click(object sender, EventArgs e)
    {
         for (int n = 0; n < 5; n++)  //multiple requests
            {
                String a = AsyncTest().Result;
            }

    }

    private async Task<String> AsyncTest()
    {
        String b = await MakeRequestAsync(url);
        return b;
    }

Also any suggestions on how could I speed this process would be great.

Answer 1

You should use Microsoft's Reactive Framework (aka Rx) - NuGet System.Reactive and add using System.Reactive.Linq; - then you can do this:

var query =
    from n in Observable.Range(0, 5)
    from r in Observable.Using(
        () => new WebClient(),
        wc => Observable.Start(() => wc.DownloadString($"{url}{n}")))
    select new { n, r };

IDisposable subscription =
    query
        .ToArray()
        .Select(xs => xs.OrderBy(x => x.n).Select(x => x.r).ToArray())
        .Subscribe(rs =>
        {
            /* Do something with the results */
        });

It's all multi-threaded and non-blocking.

If you don't care about the order in which the results are returned then just do this:

var query =
    from n in Observable.Range(0, 5)
    from r in Observable.Using(
        () => new WebClient(),
        wc => Observable.Start(() => wc.DownloadString($"{url}{n}")))
    select r;

IDisposable subscription =
    query
        .Subscribe(r =>
        {
            /* Do something with each result */
        });

Calling subscription.Dispose() will stop the query if you need to cancel it.

---

You can write your MakeRequestAsync method like this:

public static async Task<String> MakeRequestAsync(String url)
{
    return await
        Observable.Using(
            () => new WebClient(),
            wc => Observable.Start(() => wc.DownloadString(url)));
}