how to get the original mpi rank from split communicators

Question

I split the world rank to the different commiunicators

    MPI_Comm_split(world_comm, color_, key_worker,
                   &color_comm_worker);
    MPI_Comm_split(world_comm, color_master, key_master, &color_comm_master);
  
    int color_worker_size, color_master_size;
    MPI_Comm_size(color_comm_worker, &color_worker_size);
    MPI_Comm_size(color_comm_master, &color_master_size);
  
    int color_worker_rank, color_master_rank;
    MPI_Comm_rank(color_comm_worker, &color_worker_rank);
    MPI_Comm_rank(color_comm_master, &color_master_rank);      

for example for 5 ranks I would have:

  world_rank : 1 color_worker : 1 color_master: 0 key_master: 1
  world_rank : 2 color_worker : 1 color_master: 1 key_master: 1
  world_rank : 3 color_worker : 1 color_master: 2 key_master: 1
  world_rank : 4 color_worker : 2 color_master: 0 key_master: 2
  world_rank : 0 color_worker : 0 color_master: 0 key_master: 0

as you can see color_comm_master has its own rank number started from 0. the rank 1 in this communicator is rank 2 in world communicator. How can I determine the rank id of split rank in world communicator by knowing the id of the split communicator?

Answer 1

Let's start with communicator A containing ranks 0 1 2 3 4 5

You already know that the 'color' determines which processes end up in which communicator, so if we give the first three processes one color and the next three a different color, we end up with two communicators:

The key tells the MPI implementation where to put the processes in the new communicator. for example:

MPI_Comm_split(A, rank < (nprocs/2), nprocs-rank, &newcomm);

you'll end up with two communicators: some processes will be in a communicator with ranks 0 1 2, others in a communicator with ranks 3 4 5

in these new communicators, how are the processes assigned ranks? sorted by 'key'. In this case, I've constructed a key to put the ranks in reverse order. Rank 0 has a key of '6', so it is now rank 2 in the new communicator. Rank 5 has a key of 0, so it will now be rank 0 in the new communicator.

Here's some code you can play with to clarify:

#include <mpi.h>
#include <stdio.h>

int main(int argc, char **argv) {
    int rank, new_rank, nprocs, new_nprocs;
    MPI_Comm newcomm;

    MPI_Init(&argc, &argv);
    MPI_Comm_rank(MPI_COMM_WORLD, &rank);
    MPI_Comm_size(MPI_COMM_WORLD, &nprocs);

    MPI_Comm_split(MPI_COMM_WORLD, rank < (nprocs/2), nprocs-rank, &newcomm);

    MPI_Comm_rank(newcomm, &new_rank);
    MPI_Comm_size(newcomm, &new_nprocs);

    printf("Hello from rank %d of %d (was rank %d of %d)\n",
            new_rank, new_nprocs, rank, nprocs);

    MPI_Comm_free(&newcomm);
    MPI_Finalize();
}