Search code examples
c#asynchronousdotnet-httpclient

HttpClient async - Too fast skipping webservice requests


I have an issue where i loop over about 31 web service URLs. If I put a Thread.Sleep(1000) in the top code, it will work perfectly, but if I remove this, I only get success on 10 (sometimes less and sometimes more) request out of 31. How do I make it wait?

Code

foreach(var item in ss)
{ 
   //Call metaDataApi(url,conn,name,alias)
}

 public static void metadataApi(string _url, string _connstring, string _spname, string _alias)
        {
           // Thread.Sleep(1000);
            //Metadata creation - Table Creation
            using (var httpClient = new HttpClient())
            {
                string url = _url;

                using (HttpResponseMessage response = httpClient.GetAsync(url).GetAwaiter().GetResult())
                using (HttpContent content = response.Content)
                {
                    Console.WriteLine("CHECKING");
                    if (response.IsSuccessStatusCode)
                    {
                        Console.WriteLine("IS OK");
                        string json = content.ReadAsStringAsync().GetAwaiter().GetResult();

                       //Doing some stuff not relevant

                     }
                }
            }
         }

How it can look enter image description here


Solution

  • You should look to use async/await where you can, but you could try something like this:

    // you should share this for connection pooling  
    public static HttpClient = new HttpClient();
    
    public static void Main(string[] args)
    {
        // build a list of tasks to wait on, then wait
        var tasks = ss.Select(x => metadataApi(url, conn, name, alias)).ToArray();
        Task.WaitAll(tasks);
    }
    
    public static async Task metadataApi(string _url, string _connstring, string _spname, string _alias)
    {
        string url = _url;
        var response = await httpClient.GetAsync(url);
        Console.WriteLine("CHECKING");
        if (response.IsSuccessStatusCode)
        {
            Console.WriteLine("IS OK");
            string json = await content.ReadAsStringAsync();
    
            //Doing some stuff not relevant
    
        }
    }
    

    One thing to note, this will try to run many in parallel. If you need to run them all one after the other, may want to make another async function that waits on each result individually and call that from the Main. .Result is a bit of an antipattern (with modern c# syntax, you can use async on the main function) but for your script it should be "ok", but I'd minimize usage of it (hence why I wouldn't use .Result inside of a loop.