Search code examples
c#databasemultithreadingmulticore

C# - Multicore threading for database intensive app

I have an application, which need to do some validation on the database end. Each process goes through database validation (search on the table that has around 6 million records), then goes through some Lucene Index searcher (the index is built off this 6 million record table). As these steps are disjoint for each line items being passed, I am thinking of utilizing multicore threading. (each of these lines take around 1/2 minute on a single thread).

What are my options with multicore in C#? Is there some good resources / third party library (I looked a bit at PowerThreading by Jeff Ritcher), some good tutorials.

I assume I need to do some thread pools in N core machines.

Currently, it takes around 40 secs to process 100 lines, looking to get this done to around 10 secs.

Thanks...

Solution

  • Simple threading should give you access to the multicore. You will have to play around with the size of the thread pool, as your tasks look to have a lot of IO as well.