How to iterate over multiple span-s in Parallel.ForEach?

Question

input code:

code I have now:

for (int i4 = 0; i4 < ba_result_out.Length; i4 += c.i_key_size)
{
  k = BitConverter.ToInt64(spn_data.Slice(i4, c.i_key_size));
  if (Dictionary.md1.ContainsKey(k)) { 
    //some logics skipped
  }
}

code I'm trying to make: (based on: https://learn.microsoft.com/en-us/dotnet/standard/parallel-programming/how-to-speed-up-small-loop-bodies )

ParallelOptions po = new ParallelOptions { MaxDegreeOfParallelism = 2 };
var rp = Partitioner.Create(0, ba_result_out.Length / c.i_key_size, po.MaxDegreeOfParallelism);
Parallel.ForEach(rp, po, (range, loopState) =>
{
  for (int i4 = range.Item1; i4 < range.Item2; i++)
  {
  k = BitConverter.ToInt64(spn_data.Slice(i4, c.i_key_size));
  if(Dictionary.ContainsKey(k)){
    //some logics skipped
  }
});

task: make it Parallel.ForEach, not possible with span.

problem: compiler does not allow span in lambda

Is it possible to loop via multiple spans in parallel for each of them?

n.b. this is very hot code - billions of iterations - so allocation is not an option - need to stick to spans.

Answer 1

Thank you to those who participated!

I made it work as intended, using AsSpan() inside lambda function:

1. I switched to artificial array of indexes instead of spans as a base for PFE
2. i used 1 allocation for Span inside lambda (as whole indexes are / by number of cores in partitioner it makes only 4 allocations of span)
3. this is an implementation of small-body-loop parallelization from MS (link my original post)
4. it can be further improoved via passing the pointer to span thus avoiding allocation as mentioned here

I ended up with this one:

i_max_search_threads = 4;
int[] ia_base_idxs = Enumerable.Range(0, ba_result_out.Length).ToArray();
var rp = Partitioner.Create(0, ia_base_idxs.Length, ia_base_idxs.Length / i_max_search_threads);
Parallel.ForEach(rp, po, (range, loopState) =>
{
 Span<byte> spn_data = ba_result_out.AsSpan();
 for (int i4 = range.Item1; i4 < range.Item2; i4 += c.i_key_size)
 {
   k = BitConverter.ToInt64(spn_data.Slice(i4, c.i_key_size));
   if(Dictionary.ContainsKey(k)){
       //some logics skipped...
     }
   }
});