Search code examples
c#multithreadingfilethread-safetythreadpool

Writing to a File using threading or queue in c#


I have following code where its taking lot of time.I was thinking of having 4 threads which would read ids from file and then dump the details which is done by ConnectToServiceAndDump.Since all the ids are unique I am ok wih 4 threads picking up 4 unique id and making call to ConnectToServiceAndDump. I have tried Threading class to set the number of threads to 4 ,but that doesn't work if I keep it in the while loop.I am still new to threading so not sure whats the right approach.

Or an approach something like this:

//- create a list.. which contains all the ids
//- create a new thread 
//- in a theard safe way pop a loan from list    
//- dump this id by calling service in the same thread 
//- use ConcurrentQueue(T)  and Enqueue

using (StreamReader sr = new StreamReader($@"{File}"))
{
    var ids = sr.ReadLine();
    while (ids != null)
    {                                
        ConnectToServiceAndDump(client, ids, outputSubdirectoryName);
        ids = sr.ReadLine();
    }
}

Solution

  • var ids = new List<string>();
    using (StreamReader sr = new StreamReader($@"{File}"))
    {
        var id = sr.ReadLine();
        while (id != null)
        {
            ids.Add(id);
            id = sr.ReadLine();
        }
    }
    Parallel.ForEach(ids, (id) => ConnectToServiceAndDump(client, id, outputSubdirectoryName));