Parallel.ForEach with modulo partitioning

Question

When using Parallel.ForEach() for 100 items using 4 threads, it will divide the list into 4 blocks (0-24, 25-49, 50-74, 75-99) of items, which means, that items 0, 25, 50 and 75 are processed in parallel.

Is it somehow possible to partition the items in a modulo way to handle those with lower indices first? Like:

Thread 1: 0, 5, 9,...
Thread 2: 1, 6, 10,...
Thread 3: 2, 7, 11,...
Thread 4: 3, 8, 12,...

Answer 1

This partitioning method is known as Round Robin, or Striping. The primary challenge of using this with Parallel.ForEach() is that ForEach() requires partitioners to support dynamic partitions, which would not be possible with this type of partitioning as the number of partitions must be fixed prior to execution of the loop.

One way to achieve this type of partitioning is to create a custom class derived from System.Collections.Concurrent.Partitioner<TSource> and use the ParallelQuery.ForAll() method, which does not have the dynamic partitioning support requirement. For most applications this should be equivalent to using ForEach().

Below is an example of a custom Partitioner and a basic implementation. The Partitioner will generate the same number of partitions as the degree of parallelism.

using System;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.Linq;

namespace RoundRobinPartitioning
{
    public class RoundRobinPartitioner<TSource> : Partitioner<TSource>
    {
        private readonly IList<TSource> _source;

        public RoundRobinPartitioner(IList<TSource> source)
        {
            _source = source;
        }

        public override bool SupportsDynamicPartitions { get { return false; } }

        public override IList<IEnumerator<TSource>> GetPartitions(int partitionCount)
        {
            var enumerators = new List<IEnumerator<TSource>>(partitionCount);

            for (int i = 0; i < partitionCount; i++)
            {
                enumerators.Add(GetEnumerator(i, partitionCount));
            }

            return enumerators;
        }

        private IEnumerator<TSource> GetEnumerator(
            int partition,
            int partitionCount)
        {
            int position = partition;
            TSource value;

            while (position < _source.Count)
            {
                value = _source[position];
                position += partitionCount;
                yield return value;
            }
        }
    }

    class Program
    {
        static void Main(string[] args)
        {
            var values = Enumerable.Range(0, 100).ToList();

            var partitioner = new RoundRobinPartitioner<int>(values);

            partitioner.AsParallel()
                .WithDegreeOfParallelism(4)
                .ForAll(value =>
                {
                    // Perform work here
                });
        }
    }
}