I have a list of items and for each item i need to execute several async API requests and process the responses, i have seen several implementation and they are all similar when it comes to execution time but i would like to know the difference between them.
Approach 1
Parallel.ForEach(items, new ParallelOptions { MaxDegreeOfParallelism = 10 }, item =>
{
var response = item.propertyOne.GetAsync().GetAwaiter().GetResult();
//process it
var response = item.propertyTwo.GetAsync().GetAwaiter().GetResult();
//process it
var response = item.propertyThree.GetAsync().GetAwaiter().GetResult();
//process it
});
Approach 2
Parallel.ForEach(items, new ParallelOptions { MaxDegreeOfParallelism = 10 }, item =>
{
Task.Run(async () =>
{
var response = await item.propertyOne.GetAsync();
}).GetAwaiter().GetResult();
//process it
Task.Run(async () =>
{
var response = await item.propertyTwo.GetAsync();
}).GetAwaiter().GetResult();
//process it
Task.Run(async () =>
{
var response = await item.propertyThreee.GetAsync();
}).GetAwaiter().GetResult();
//process it
});
Approach 3 assume some bounding mechanism is applied in order not to flood the system with tasks
List<Task> tasksToAwait = new List<Task>();
foreach (var item in items)
{
tasksToAwait.Add(Task.Run(async () =>
{
var response = await item.propertyOne.GetAsync();
//process it
var response = await item.propertyTwo.GetAsync();
//process it
var response = await item.propertyThree.GetAsync();
//process it
}));
}
await Task.WhenAll(taskToAwait);
Notes:
Which one of these is recommended and if none what do you suggest? Also, how are they different and why is there an execution time difference?
Since your methods are asynchronous, just call them all but instead of awaiting them individually, await all of them to make sure that all three asynchronous processes completed. Then, you can use await
again to unwrap the results:
// start all asynchronous processes at once for all three properties
var oneTask = item.propertyOne.GetAsync();
var twoTask = item.propertyTwo.GetAsync();
var threeTask = item.propertyThree.GetAsync();
// await the completion of all of those tasks
await Task.WhenAll(oneTask, twoTask, threeTask);
// access the individual results; since the tasks are already completed, this will
// unwrap the results from the tasks instantly:
var oneResult = await oneTask;
var twoResult = await twoTask;
var threeResult = await threeTask;
In general, when dealing with asynchronous stuff, avoid calling .GetResult()
, .Result
, or .Wait()
at all costs. Neither of these will do anything good if you have an asynchronous process.
To reply to your comments:
I assume the code would go inside the parallel foreach right?
No, you wouldn’t use Parallel.ForEach
here because you are still calling asynchronous processes. Parallel.ForEach
is for loading work off to multiple threads but that’s not what you want to do here. You have asynchronous processes that can all run concurrently without requiring additional threads.
So if you have a lot of these things that you want to call asynchronously, just call them all and collect the Task
s to await them later.
But what if the requests had to be sequential such as paged responses where the first request would return a URL for the next page request and so on?
Then you will need to make the calls depend on each other for which you use await
to make sure that an asynchronous process is completed before you can continue. For example, if the propertyTwo
calls depend on propertyOne
, then you can still make all propertyOne
calls first and only continue when these are complete.
If you split up your logic into separate methods, you will notice that this becomes easier:
var itemTasks = items.Select(item => GetPropertiesOneTwoThreeAsync(item));
await Task.WhenAll(itemTasks);
private Task GetPropertiesOneTwoThreeAsync(Item item)
{
// get the properties sequentially
var oneResult = await item.propertyOne.GetAsync();
var twoResult = await item.propertyTwo.GetAsync();
var threeResult = await item.propertyThree.GetAsync();
}
Also, if the calling method is not async, would
Task.WhenAll(oneTask, twoTask, threeTask).GetAwaiter().GetResult()
be appropriate in this case?
No. Calling asynchronous methods from synchronous methods is generally not something that you should do. If you call asynchronous methods, you should make the calling method also asynchronous. There a ways to make this work but you will have to deal with potential deadlocks then and should generally know what you are doing. See also this question.