Serialize mpi threads

Question

Is there anyway to serialize a certain part of your MPI code ? For example when printing the information out to the screen. Something like below:

MPI_SERIALIZE();

cerr << "THIS WILL BE PRINTED ";
cerr << "IN ORDER" << endl;

MPI_END_SERILIZE();

If there are two MPI threads, there will be no case :

THIS WILL BE PRINTED THIS WILL BE PRINTED IN ORDER 
IN ORDER

Thanks

Answer 1

The way I do this in demonstration programs (and note - you would only do this in little demo programs because of the high synchronization cost; if you're trying to control output to a data file, you'd use MPI-IO, and if you're trying to coordinate output to the terminal, easiest to send data to task 0 and have it do all the output) is to loop over barriers, something like this:

#include <iostream>
#include <mpi.h>

using namespace std;

int main(int argc, char **argv) {

    int rank, size;
    int ierr;

    ierr = MPI_Init(&argc, &argv);

    ierr = MPI_Comm_size(MPI_COMM_WORLD, &size);
    ierr = MPI_Comm_rank(MPI_COMM_WORLD, &rank);

    for (int i=0; i<size; i++)
    {
        if (i == rank) {
            cout << "Hello from task " << rank << " of "
                 << size << " world!" << endl;
        }
        MPI_Barrier(MPI_COMM_WORLD);
    }

    MPI_Finalize();

    return 0;
}

(And as a smaller thing, MPI doesn't have threads, it has processes. That may seem like a small detail, but if you start combining MPI with OpenMP, for instance, the distinction between threads and processes becomes important.)