Search code examples
c#multithreadingthread-safetyrestsharpparallel.foreach

Using Parallel.ForEach Create multiple requests in parallel and put them in the list


So I had to create dozens of API requests and get json to make it an object and put it in a list. I also wanted the requests to be parallel because I do not care about the order in which the objects enter the list.

  public ConcurrentBag<myModel> GetlistOfDstAsync()
  {
   var req = new RequestGenerator();
   var InitializedObjects = req.GetInitializedObjects();
   var myList = new ConcurrentBag<myModel>();

   Parallel.ForEach(InitializedObjects, async item =>
            {
            RestRequest request = new RestRequest("resource",Method.GET);
            request.AddQueryParameter("key", item.valueOne);
            request.AddQueryParameter("key", item.valueTwo);


            var results = await GetAsync<myModel>(request);

            myList.Add(results);
            });

   return myList;
   }

What creates a new problem, I do not understand how to put them in the list and it seems I do not use a solution that exists in a form ConcurrentBag

Is my assumption correct and I implement it wrong or should I use another solution?


Solution

  • I also wanted the requests to be parallel

    What you actually want is concurrent requests. Parallel does not work as expected with async.

    To do asynchronous concurrency, you start each request but do not await the tasks yet. Then, you can await all the tasks together and get the responses using Task.WhenAll:

    public async Task<myModel[]> GetlistOfDstAsync()
    {
      var req = new RequestGenerator();
      var InitializedObjects = req.GetInitializedObjects();
      var tasks = InitializedObject.Select(async item =>
      {
        RestRequest request = new RestRequest("resource",Method.GET);
        request.AddQueryParameter("key", item.valueOne);
        request.AddQueryParameter("key", item.valueTwo);
        return await GetAsync<myModel>(request);
      }).ToList();
    
      var results = await TaskWhenAll(tasks);
      return results;
    }