i ran into a problem with the flag -Dparallel (which we use in our code to enable parallel parts). When i use it, i dont get more then one OpenMP Thread anymore.
Remark: We are running MPI/OpenMP hyrbrid code.
A short code example:
#include <stdio.h>
#include "mpi.h"
#ifdef _OPENMP
#include <omp.h>
#else
#define omp_get_num_threads() 0
#define omp_get_thread_num() 0
#endif
int main(int argc, char **argv)
{
int nthreads, thread_id;
int rank, size, provided;
MPI_Init_thread(&argc, &argv, MPI_THREAD_FUNNELED, &provided); /* starts MPI */
MPI_Comm_rank (MPI_COMM_WORLD, &rank); /* get current process id */
MPI_Comm_size (MPI_COMM_WORLD, &size); /* get number of processes */
#pragma omp parallel private(nthreads, thread_id)
{
nthreads = omp_get_num_threads();
thread_id = omp_get_thread_num();
if (thread_id == 0)
printf("This is rank %d with %d threads.\n", rank,nthreads);
}
MPI_Finalize();
return 0;
}
If i compiled it with mpicc -o example -fopenmp example.c the output shows:
$ mpirun -np 2 ./example
This is rank 1 with 6 threads.
This is rank 0 with 6 threads.
As expected.
If i use the flag -Dparallel (whole line: mpicc -Dparallel -o example_parallel -fopenmp example.c )
The output says:
$ mpirun -np 2 ./example_parallel
This is rank 0 with 1 threads.
This is rank 1 with 1 threads.
Why does -Dparallel limits the OpenMP Threadfs to 1? Where can i find documentation for this?
for the sake of completeness:
EDIT (maybe the solution): I just testet to compile it using:
mpicc -Dparallel=parallel -o example_parallel -fopenmp example.c
it works as expected. So i guess, -Dparallel just messes up the #pragma omp parallel. Am i right?
With most C compilers, -D is an option to define the value of a preprocessor macro symbol. With -Dparallel you define a preprocessor symbol called parallel but with an empty value. As preprocessing is always done before the code is further parsed by the compiler, all occurrences of parallel in your code are replaced by an empty string and #pragma omp parallel private(...) becomes #pragma omp private(...). The compiler likely ignores the resulting invalid pragma.
Drop -Dparallel from the command line. If you need to be able to compile different parts of the code conditionally, then use another preprocessor symbol, for example BUILD_PARALLEL, e.g.:
#if defined(BUILD_PARALLEL)
... parallel code ...
#else
... serial code ...
#endif
Never use preprocessor symbols with names that match language keywords. Using -Dparallel=parallel is an awful hack to both have the parallel preprocessor symbol defined and still have it retain its original meaning.