Using MPI_Scatter and MPI_Gather to implement Master-Worker-Model

Question

I have written a MPI program for an image processing filter. The main goal of this program is that a given filter is applied to an image file using parallelized computations.

The MPI programm works fine and there are no issues.

Now I want to use the Master-Worker-Model for my MPI program. After doing some research, I found out that MPI_Scatter and MPI_Gather are the commands I need.

But I don't quite see how they work. This is my first try to code it:

#include <mpi.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

int main(int argc, char** argv)
{

    // Filters

    MPI_Init(&argc, &argv);

    if(parent == MPI_COMM_NULL)
    {
        MPI_Scatter(); // What are the parameters?

        // Master reads in the image file and distribute it to the workers.
        MPI_File_open();
        MPI_File_read();
        MPI_File_close();
    }
    else
    {
        // After receiving data from the master, worker processes begin their job.

        /* Normally here would be the entire logic of this program.*/
        /* For simplicity I just skip it for readability.*/
        /* The calculcation is exclusively up to the workers.*/
    }

    // After finishing, the results will be sent back to the master.

    MPI_Gather(); // What are the parameters?

    MPI_Finalize(); 
    return 0; 
}

It won't be running because it's just a first prototype. What I want to know is whether I have understood the concept of the Master-Worker-Model.

Some feedback to the code would be helpful.

Thanks in advance.

Answer 1

Here is a sample program that shows how to perform the scatter/gather operations. Note in your case, since the master task is not doing any work, an inter communicator is the best fit (even if the semantic of collective operations on an inter communicator is not always very intuitive) and hence you do not need to MPI_Intercomm_merge().

#include <stdio.h>
#include <stdbool.h>

#include <mpi.h>

int main(int argc, char *argv[]) {
    MPI_Comm parent, intercomm;
    const int slaves = 2;
    bool master;
    int data[slaves];
    int mydata;
    int i, rank, root;

    MPI_Init(&argc, &argv);
    MPI_Comm_get_parent(&parent);
    master = parent == MPI_COMM_NULL;

    if (master) {
        int errcodes[slaves];
        /* spawn the slaves */
        MPI_Comm_spawn(argv[0], MPI_ARGV_NULL, slaves, MPI_INFO_NULL, 0, MPI_COMM_WORLD, &intercomm, errcodes);

        /* prepare data to be scattered to the slaves */
        for (i=0; i<slaves; i++) {
            data[i] = i;
        }

        /* here we assume there is only one master */
        root = MPI_ROOT;
    } else {
        intercomm = parent;
        root = 0;
    }

    /* scatter data from master to slaves */
    MPI_Scatter(data, 1, MPI_INT, &my-data, 1, MPI_INT, root, intercomm);

    if (!master) {
        /* slaves do their work */
        mydata = mydata + 1;
    }

    /* gather data from slaves to master */
    MPI_Gather(&mydata, 1, MPI_INT, data, 1, MPI_INT, root, intercomm);

    if (master) {
        int i;
        for (i=0; i<slaves; i++) {
            printf("Slave %d returned %d\n", i, data[i]);
        }
    }

    MPI_Comm_disconnect(&intercomm);

    MPI_Finalize();
    return 0;
}