Search code examples
cbuffermpihpc

What is the logic of needing an MPI communicator to create a packet?


What is the logic of needing to pass an MPI communicator to create a packet?

https://www.mpich.org/static/docs/v3.2/www3/MPI_Pack.html

int MPI_Pack(
             const void *inbuf,
             int incount,
             MPI_Datatype datatype,
             void *outbuf,
             int outsize,
             int *position,
             MPI_Comm comm   /*********HERE*****/
            )

In theory one could fill a buffer with the information without needing to know where the data is being sent to. Does it mean that the resulting packet format depends on some characteristic of the communicator? which one?

There is a redundancy there, what happens if one packs with one communicator and sends the packet (buffer) with a different communicator?

#include "mpi.h"
#include <stdio.h>

int main(int argc, char *argv[])
{
    int rank, size;
    int i;
    char c[100];
    char buffer[110];
    int position = 0;
    MPI_Status status;

    MPI_Init(&argc, &argv);
    MPI_Comm_size(MPI_COMM_WORLD, &size);
    if (size < 2)
    {
        printf("Please run with 2 processes.\n");fflush(stdout);
        MPI_Finalize();
        return 1;
    }
    MPI_Comm_rank(MPI_COMM_WORLD, &rank);

    if (rank == 0)
    {
        for (i=0; i<100; i++)
            c[i] = i;
        i = 123;
        MPI_Pack(&i, 1, MPI_INT, buffer, 110, &position, MPI_COMM_WORLD);
        MPI_Pack(c, 100, MPI_CHAR, buffer, 110, &position, MPI_COMM_WORLD);
        MPI_Send(buffer, position, MPI_PACKED, 1, 100, MPI_COMM_WORLD);
    }

    if (rank == 1)
    {
        MPI_Recv(buffer, 110, MPI_PACKED, 0, 100, MPI_COMM_WORLD, &status);
        MPI_Unpack(buffer, 110, &position, &i, 1, MPI_INT, MPI_COMM_WORLD);
        MPI_Unpack(buffer, 110, &position, c, 100, MPI_CHAR, MPI_COMM_WORLD);
        printf("i=%d\nc[0] = %d\n...\nc[99] = %d\n", i, (int)c[0], (int)c[99]);fflush(stdout);
    }

    MPI_Finalize();
    return 0;
}

Solution

  • If all the MPI tasks of your communicator have the same architecture, then packing/unpacking is very straightforward. Now if the MPI tasks within the same communicator run on different arch (e.g. x86_64 and sparcv9, different endianness, different encoding of MPI_LONG_DOUBLE among other things), then the data has to be packed in a neutral format so the same packed buffer can be decoded by any task).

    From the MPI 3.1 standard page 133

    The comm argument is the communicator that will be subsequently used for sending the packed message.

    so the answer to your second question (what happens if one packs with one communicator and sends the packet (buffer) with a different communicator) is an illegal program with respect to the MPI standard has an undefined behavior.