Increment ID in ParallelQuery.ForAll Method

Question

I have a list of files that needs to be processed at the same time. I tried to use ForAll extension method of the ParallelQuery class. I don't the files to be processed in order so I used ForAll.

Here's my sample code:

List<FileInfo> files = GetFilesToProcess();

files.AsParallel().ForAll(f => { // Process file here });

It works great but now I need to generate a unique integer ID for each of the file and I not sure how to do it without changing the AsParallel.ForAll to ForEach.

I read somewhere that I need to as an Interlocked but there will still be an issue.

Hope you can give me an idea here.

Thanks!

Answer 1

You can use Interlocked.Increment to generate the ID, or you can just use the index directly:

List<FileInfo> files = GetFilesToProcess();

files.AsParallel().Select((f, i) => new {File=f, ID=i})
     .ForAll(fp => 
           {
              FileInfo file = fp.File;
              int id = fp.ID; // ID is the index in the list

              // Process file here
           });

If you want to use Interlocked.Increment, you could do:

List<FileInfo> files = GetFilesToProcess();
int globalId = -1;

files.AsParallel().ForAll(f => 
                         { 
                              // Process file here 
                              int id = Interlocked.Increment(ref globalId);
                              // use ID
                         });

That being said, if your entire goal is to do "work" on a collection, I would recommend writing this as a Parallel.For or Parallel.ForEach instead. This is much more clear, as you're not using LINQ style syntax for the sole purpose of generating side effects:

List<FileInfo> files = GetFilesToProcess();
Parallel.For(0, files.Count, i =>
{
    var file = files[i];
    // Use i and file as needed
});