Parallel.ForEachAsync help needed

Question

Ok. So I am going to start by apologizing up front as I know there have been a lot of question asked about Parallel and Async. However, even after searching I can not wrap my brain around how this should work. Coding is something I dabble in not something I do day in and day out.

I am trying to help a friend collect some historic stock data using the AlphaVantage API.

I have no issues collecting the data and saving it to the database but it can take a long time to pull 20 years of daily prices. So to avoid the GUI freezing up I made it an async function. However, Considering how often and how much data needs to be pulled it will need to be done in parallel. I am not sure how many parallel events yet but the long term goal is to have a timer in the application that fires off at intervals and calls the BatchUpdateStockHistoryAsync method and passes in a value for how many stocks to pull and update in this batch the function will then go out, query the DB and get a list 5 that have the oldest update.

For the moment I stripped all of that out to simply the question and just created a manual list of 5 stocks that get iterated through. For each stock in the list it calls another function PullAndWriteHistoricDailyData(item) that does all the work of actually reaching out to AlphaVantage and updating the DB.

After this lengthy info my question is what's the best way is to trigger multiple concurrent threads of PullAndWriteHistoricDailyData()?

I imagine I will need to also play with MaxDegreeOfParallelism to figure out what works best but I have not even gotten that far yet. I believe I would want to use Parallel.ForEachAsync maybe? but just not sure how to pull it all together.

Any help would be greatly appreciated

Misiu

    public async Task BatchUpdateStockHistoryAsync()
    {
        List<string> list = new List<string>();

        list.Add("AAPL");
        list.Add("F");
        list.Add("MSFT");
        list.Add("COKE");
        list.Add("IAG");

        foreach(string item in list)
        {
            await Task.Run(() => PullAndWriteHistoricDailyData(item));
        }

        return completed;
    }

Answer 1

EDIT I mistakenly read about Task.WhenAll vs Parallel.ForEach originally. After reading more about Parallel.ForEachAsync based on the answer by Theodore Zoulias at: is-parallel-foreachasync-a-replacement-to-a-plain-for-loop-append-to-task-list, and based on the assumption that you want to view the result of each Task as apposed to receiving a single Task from the call to Parallel.ForEachAsync, then I would propose changing to using the following:

You could switch to using a List<Task> and use await Task.WhenAll() and skip the Parallel.ForEachAsync like so:

 List<Task> myupdatetasks = new List<Task>();

 foreach(string item in list)
 {
     myupdatetasks.Add(Task.Run(() => PullAndWriteHistoricDailyData(item)));
 }

 await Task.WhenAll(myupdatetasks).ConfigureAwait(false);

if PullAndWriteHistoricDailyData() is already an async method, you don't need to use Task.Run, you could do the same as above minus the Task.Run

 List<Task> myupdatetasks = new List<Task>();

 foreach(string item in list)
 {
     myupdatetasks.Add(PullAndWriteHistoricDailyData(item));
 }

 await Task.WhenAll(myupdatetasks).ConfigureAwait(false);

if you are intent on using Parallel.ForEachAsync, this article explains it pretty well: parallelforeachasync-in-net-6