MPI - sending messages to processes that run from other functions

Question

I have a process with rank 0 (MASTER) that is running in a function (FUNCA) that does:

...

get_moves_list(node,&moves_list,&moves_len,maximizing);
//for each rank in SLAVES
//MPI_Send a move to a SLAVE

I want the slave processes to receive messages from the MASTER, but the slave processes are running from/inside a different function (FUNCB)

void run_slave(rank) {
    int move;
    //MPI_Recv a move from MASTER
    //Do some stuff with that move
    //Then return to caller 
}

Main looks like this

int main(int argc,char **argv)
{
    int rank,size;
    MPI_Init(NULL,NULL);
    MPI_Comm_rank(MPI_COMM_WORLD,&rank);
    MPI_Comm_size(MPI_COMM_WORLD,&size);

    if (rank == MASTER) {
        ...

        //CALL FUNCA

        ...
    } else {
        run_slave(rank);
        MPI_Finalize();
    }
}

Is something like this possible with MPI, sending/receiving messages to processes running in different functions?

If it helps, I am trying to parallelize a minimax function (FUNCA), but the structure of the program must be used as described above.

When the program is started MASTER process initiates a game and calls minimax to get an optimal move for the maximizing player.

I have the serial version of minimax working and am currently attempting to parallelize it using MPI with no luck so far.

Answer 1

To make it clear, MPI is a structured communication library, not some esoteric parallel programming language extension. It simply facilitates the structured exchange of data between entities called ranks. Generally, ranks are processes running on the same computer or on separate computers linked with some kind of network, but those could also be some other kind of communicating entities. What is important is that each rank is on its own when it comes to code execution and it doesn't care where in the program the other ranks are. Even more, it doesn't care if the other ranks are running the same program. In fact, although it is typical for MPI to have all ranks run the same code, the so-called SPMD or Single Program Mulitple Data, you are free to write a separate program for a group of ranks or even for each rank, which is known as MPMD or Multiple Programs Multiple Data. MPI even facilitates the classical client-server mode and allows separate MPI jobs to connect. SPMD is simply easier to program as you need to write a single program only.

Think of MPI simply as a mediator (middleware) between your code and the system-specific APIs that enables easy interprocess communication and abstracts away things such as locating the actual endpoints of the other communicating partners (e.g., finding out the network addresses and port numbers when communication runs over TCP/IP). When you write a browser that communicates over the network with a WEB server, you don't care what code the server executes. Conversely, the server doesn't care what code your browser executes. As long as both speak the same protocol, the communication works. The same applies to MPI - as long as two ranks use the same MPI library, they can communicate.

For successful communication to happen in MPI, there are only two things needed (in a typical point-to-point data exchange):

• sender: rank A is willing to send data to rank B and so it calls MPI_Send(..., B, tag, MPI_COMM_SOMETHING);
• receiver: rank B is willing to receive data from rank A and so it calls MPI_Recv(..., A, tag, MPI_COMM_SOMETHING, ...);

As long as both ranks specify the same tag and communicator and the addresses in both send and receive calls match pairwise (including the ability of the receiver to specify source and tag wildcards), the exchange will happen regardless of where the actual lines of code are located.

The following is a perfectly valid MPI example:

rank_0.c

#include <stdio.h>
#include <mpi.h>

int main(void)
{
   MPI_Init(NULL, NULL);

   int rank;
   MPI_Comm_rank(MPI_COMM_WORLD, &rank);

   int a;
   MPI_Recv(&a, 1, MPI_INT, 1, 0, MPI_COMM_WORLD, MPI_STATUS_IGNORE);
   printf("Rank %d got %d from rank 1\n", rank, a);

   MPI_Finalize();
   return 0;
}

rank_1.c

#include <mpi.h>
#include <stdio.h>

int main(int argc, char **argv)
{
   MPI_Init(&argc, &argv);

   int rank;
   MPI_Comm_rank(MPI_COMM_WORLD, &rank);

   int a = 42;
   MPI_Send(&a, 1, MPI_INT, 0, 0, MPI_COMM_WORLD);
   printf("Rank %d sent %d to rank 0\n", rank, a);

   MPI_Finalize();
   return 0;
}

Compile and run:

$ mpicc -o rank_0 rank_0.c
$ mpicc -o rank_1 rank_1.c
$ mpiexec -n 1 ./rank_0 : -n 1 ./rank_1
Rank 0 got 42 from rank 1
Rank 1 sent 42 to rank 0

As you can see, those are two completely different programs and they still happily run together in one MPI job and are able to exchange messages.