How can I paralelize two for statements equally between threads using OpenMP?

Question

Lets say I have the following code:

#pragma omp parallel for
for  (i = 0; i < array.size; i++ ) {
   int temp = array[i];
   for (p = 0; p < array2.size; p++) {
      array2[p] = array2[p] + temp;

How can I divide the array2.size between the threads that I call when I do the #pragma omp parallel for in the first line? For what I understood when I do the #pragma omp parallel for I'll spawn several threads in such a way that each thread will have a part of the array.size so that the i will never be the same between threads. But in this case I also want those same threads to have a different part of the array2.size (their p will also never be the same between them) so that I dont have all the threads doing the same calculation in the second for.

I've tried the collapse notation but it seems that this is only used for perfect for statements since I couldn't get the result I wanted.

Any help is appreciated! Thanks in advance

Answer 1

The problem with your code is that multiple threads will try to modify array2 at the same time (race condition). This can easily be avoided by reordering the loops. If array2.size doesn't provide enough parallelism, you may apply the collapse clause, as the loops are now in canonical form.

#pragma omp parallel for
for (p = 0; p < array2.size; p++) {
    for (i = 0; i < array.size; i++ ) {
        array2[p] += array[i];
    }
}

You shouldn't expect too much of this though as the ratio between loads/stores and computation is very bad. This is without a doubt memory-bound and not compute-bound.

EDIT: If this is really your problem and not just a minimal example, I would also try the following:

#pragma omp parallel 
{
    double sum = 0.;
    #pragma omp for reduction(+: sum)
    for (i = 0; i < array.size; i++) {
        sum += array[i];
    }
    #pragma omp for
    for (p = 0; p < array2.size; p++) {
        array2[p] += sum;
    }
}