I'm executing a query that returns approximately 11,000 rows. Each one of these rows is mapped to a unique POCO that I wrote a lightweight ORM to handle creating, called ToCustomObject<T>
. Right now, I have an already async method that gets the results of this query and puts them into a DataTable
. I then loop through all of the rows and call ToCustomObject
on them to generate my object, and then add it to a List<T>
. The query itself only takes half a second, but the loop to generate all my POCOs takes multiple seconds.
public async Task<List<Foo>> GetFoosAsync()
{
//empty list to populate and return
List<Foo> foos = [];
//example, this is not how the query actually executes, do not need assistance with this part
string query = "select t.Field1, t.Field2 from Table";
DataTable results = await customDbHandlerThatsNotRelevant.ExecuteQueryAsync(query);
//async way to handle this?
foreach (DataRow row in results.Rows)
foos.Add(row.ToCustomObject<Foo>());
return foos ;
}
I had thought about maybe using Task.WhenAll()
to put all the generations into a task list and execute that way, but foos.Add()
is not awaitable. I also looked into Parallel.ForEach()
, but I'm not as familiar with that and from what I could tell, would not accomplish what I'm trying to do.
TL;DR: How can I make all occurrences of foos.Add(row.ToCustomObject<Foo>());
occur asynchronously?
Well it greatly depends. Are you seeing performance bottleneck in that loop? If yes, then maybe it is worth parallelizing.
But generally, it is not meant to be executed asynchronously, as your method ToCustomObject
is not async, so you would need to wrap each execution in Task
, which is already overhead. Doing that for all item in a list may even downgrade your performance!
BUT, if you are seeing performance issues, I can suggest couple of ways:
ToCustomObject
call in Task
, creating Task
for each item and awaiting them all at oneParallel.ForEach
ToCustomObject
is performing badly, maybe there's some synchornous code inside of it that can be done asynchoronously? Then you could make async version of ToCustomObject
Of course, you need to take care of your List
beong accessed potentially from multiple threads, do you need to use some synchronization or threads safe collection.