How to print something from processes without other processes overlapping it?

I have written an MPI code which I am running using 16 processes and I am printing 4x4 matrix as output from each of those process:

printf("\n This is the output of %d\n",myrank);
for(int i=0;i<count;i++)
{
    for(int j=0;j<count;j++)
    {
        printf("%f ",m[i][j]);
    }
    printf("\n");
}

But the problem is that the outputs from each process gets mixed up with others. Like this--

 This is the output of 3
0.750000 0.750000 0.750000 0.500000 
 This is the output of 9
1.000000 1.000000 1.000000 1.000000 
1.000000 1.000000 1.000000 
1.000000 1.000000 1.000000 0.750000 
1.000000 1.000000 1.000000 0.750000 1.000000 
1.000000 1.000000 1.000000 1.000000 
1.000000 1.000000 1.000000 1.000000 
1.000000 1.000000 1.000000 0.750000

How to prevent this kind of behaviour?

Solution

I have written an MPI code which I am running using 16 processes and I am printing 4x4 matrix as output from each of those process.

This is not how MPI is meant to be used, actually parallelism in general IMO. The coordination of printing the output to the console among processes will greatly degrade the performance of the parallel version.

Most of the times one is better off just making one process responsible for printing the output to the console (typically the master process i.e., process with rank = 0).

Nonetheless, you can try something like the following with the MPI_Barrier, like so:

int rank;
MPI_Comm_rank (MPI_COMM_WORLD, &rank);  /* get current process id */
...
MPI_Barrier(MPI_COMM_WORLD);
if(rank == /** the rank of the process that will print the output **/)
{
   // printing the data to the output
}
MPI_Barrier(MPI_COMM_WORLD);

For your case:

MPI_Barrier(MPI_COMM_WORLD);
printf("\n This is the output of %d\n",myrank);
MPI_Barrier(MPI_COMM_WORLD);
for(int i=0;i<count;i++)
{
    for(int j=0;j<count;j++)
    {
        printf("%f ",m[i][j]);
    }
    printf("\n");
}

This would at least avoid the output "This is the output of 3" to be mixed with the the output of the matrices.

Bear in mind, however that (quoted from the comment kindly provided by Hristo Iliev):

Using barriers like that only works for local launches when (and if) the processes share the same controlling terminal. Otherwise, it is entirely to the discretion of the I/O redirection mechanism of the MPI implementation.

To print the matrix in order would require a more complex mechanism where you would use MPI_Send and MPI_Recv. Could be something like the processes wait on a MPI_Recv for the another process to send a message signaling that this process just finished printing its part of the matrix. For example, with 4 processes:

Process 1 waits for the message of process 0, process 2 waits the message of process 1 and so on. Process 0 print part of the matrix and send the message to process 1 which proceeds in the same way. But again you are better off sending the entire matrix to process 0 and having that process printing the matrix to the console.