I would like to call my API in parallel x number of times so processing can be done quickly.
I have three methods below that I have to call APIs in parallel. I am trying to understand which is the best way to perform this action.
Base Code:
var client = new System.Net.Http.HttpClient();
client.DefaultRequestHeaders.Add("Accept", "application/json");
client.BaseAddress = new Uri("https://jsonplaceholder.typicode.com");
var list = new List<int>();
var listResults = new List<string>();
for (int i = 1; i < 5; i++)
{
list.Add(i);
}
First method using Parallel.ForEach:
Parallel.ForEach(list,new ParallelOptions() { MaxDegreeOfParallelism = 3 }, index =>
{
var response = client.GetAsync("posts/" + index).Result;
var contents = response.Content.ReadAsStringAsync().Result;
listResults.Add(contents);
Console.WriteLine(contents);
});
Console.WriteLine("After all parallel tasks are done with Parallel for each");
Second method with Tasks. I am not sure if this runs parallel. Let me know if it does:
var loadPosts = new List<Task<string>>();
foreach(var post in list)
{
var response = await client.GetAsync("posts/" + post);
var contents = response.Content.ReadAsStringAsync();
loadPosts.Add(contents);
Console.WriteLine(contents.Result);
}
await Task.WhenAll(loadPosts);
Console.WriteLine("After all parallel tasks are done with Task When All");
Third method using Action Block - This is what I believe I should always do but I want to hear from community:
var responses = new List<string>();
var block = new ActionBlock<int>(
async x => {
var response = await client.GetAsync("posts/" + x);
var contents = await response.Content.ReadAsStringAsync();
Console.WriteLine(contents);
responses.Add(contents);
},
new ExecutionDataflowBlockOptions
{
MaxDegreeOfParallelism = 6, // Parallelize on all cores
});
for (int i = 1; i < 5; i++)
{
block.Post(i);
}
block.Complete();
await block.Completion;
Console.WriteLine("After all parallel tasks are done with Action block");
Approach number 2 is close. Here's a rule of thumb: I/O bound operations=> use Tasks/WhenAll (asynchrony), compute bound operations => use Parallelism. Http Requests are network I/O.
foreach (var post in list)
{
async Task<string> func()
{
var response = await client.GetAsync("posts/" + post);
return await response.Content.ReadAsStringAsync();
}
tasks.Add(func());
}
await Task.WhenAll(tasks);
var postResponses = new List<string>();
foreach (var t in tasks) {
var postResponse = await t; //t.Result would be okay too.
postResponses.Add(postResponse);
Console.WriteLine(postResponse);
}