Say I have n processes:
They do a calculation, and then send the result to rank 0. This is what I want to happen:
Rank 0 will wait till it has a result from all of the ranks and then add them up.
How do I do this? Also, I want to avoid the following:
For eg. 4 processes P0, P1, P2, P3,
P1 -> P0
P2 -> P0
P3 -> P0
In the meanwhile P1 has finished its calculation and so P1->P0 happens again.
I want P0 to only make the addition for 3 processes in one cycle before doing it for the next one.
Can someone suggest an MPI function to do this? I know about MPI_Gather but I'm not sure if its blocking
I've thought up of this:
#include <mpi.h>
#include <stdio.h>
int main()
{
int pross, rank,p_count = 0;
int count = 10;
MPI_Init(&argc,&argv);
MPI_Comm_size(MPI_COMM_WORLD,&pross);
MPI_Comm_rank(MPI_COMM_WORLD,&rank);
int * num = malloc((pross-1)*sizeof(int));
if(rank !=0)
{
MPI_Send(&count,1,MPI_INT,0,1,MPI_COMM_WORLD);
}
else
{
MPI_Gather(&count, 1,MPI_INT,num, 1, MPI_INT, 0,MPI_COMM_WORLD);
for(ii = 0; ii < pross-1;ii++ ){printf("\n NUM %d \n",num[ii]); p_count += num[ii]; }
}
MPI_Finalize();
}
I'm getting error:
*** Process received signal ***
Signal: Segmentation fault (11)
Signal code: Address not mapped (1)
Failing at address: (nil)
[ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x11630)[0x7fb3e3bc3630]
[ 1] /lib/x86_64-linux-gnu/libc.so.6(+0x90925)[0x7fb3e387b925]
[ 2] /usr/lib/libopen-pal.so.13(+0x30177)[0x7fb3e3302177]
[ 3] /usr/lib/libmpi.so.12(ompi_datatype_sndrcv+0x54c)[0x7fb3e3e1e3ec]
[ 4] /usr/lib/openmpi/lib/openmpi/mca_coll_tuned.so(ompi_coll_tuned_gather_intra_basic_linear+0x143)[0x7fb3d53d9063]
[ 5] /usr/lib/libmpi.so.12(PMPI_Gather+0x1ba)[0x7fb3e3e29a3a]
[ 6] sosuks(+0xe83)[0x55ee72119e83]
[ 7] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf1)[0x7fb3e380b3f1]
[ 8] sosuks(+0xb5a)[0x55ee72119b5a]
*** End of error message ***
Also, I tried:
#include <mpi.h>
#include <stdio.h>
int main()
{
int pross, rank,p_count = 0;
int count = 10;
MPI_Init(&argc,&argv);
MPI_Comm_size(MPI_COMM_WORLD,&pross);
MPI_Comm_rank(MPI_COMM_WORLD,&rank);
int * num = malloc((pross-1)*sizeof(int));
if(rank !=0)
{
MPI_Send(&count,1,MPI_INT,0,1,MPI_COMM_WORLD);
}
else
{
MPI_Gather(&count, 1,MPI_INT,num, 1, MPI_INT, 0,MPI_COMM_WORLD);
for(ii = 0; ii < pross-1;ii++ ){printf("\n NUM %d \n",num[ii]); p_count += num[ii]; }
}
MPI_Finalize();
}
I'm getting error here:
*** Process received signal ***
Signal: Segmentation fault (11)
Signal code: Address not mapped (1)
Failing at address: 0x560600000002
[ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x11630)[0x7fefc8c11630]
[ 1] mdscisuks(+0xeac)[0x5606c1263eac]
[ 2] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf1)[0x7fefc88593f1]
[ 3] mdscisuks(+0xb4a)[0x5606c1263b4a]
*** End of error message ***
For the second attempt, the thing to note here is that the send and recv were successful but root could only receive 2 messages from ranks for some reason. The segmentation fault seen was due to there being only two elements in num, I dont see why num only receives twice.
I'm calling the code as
mpiexec -n 6 ./sosuks
Can someone tell me a better / correct way to implement my idea ?
UPDATE:
Apart from the answer below I found the mistake in my implementation above which I wanted to share:
#include <mpi.h>
#include <stdio.h>
int main()
{
int pross, rank,p_count = 0;
int count = 10;
MPI_Init(&argc,&argv);
MPI_Comm_size(MPI_COMM_WORLD,&pross);
MPI_Comm_rank(MPI_COMM_WORLD,&rank);
MPI_Status status;
int * num = malloc((pross-1)*sizeof(int));
if(rank !=0)
{
MPI_Send(&count,1,MPI_INT,0,1,MPI_COMM_WORLD);
}
else
{
int var,lick = 0;
for(lick = 1; lick < pross; u++)
{
int fetihs;
MPI_Recv(&fetihs,1,MPI_INT,lick,1,MPI_COMM_WORLD,&status);
var += fetihs;
}
// do things with var
}
MPI_Finalize();
}
In your case, as Sneftel pointed out, you need MPI_Reduce. Also, you don't need explicit synchronization before the cycle completes.
#include <mpi.h>
#include <stdio.h>
#include <stdlib.h>
int main(int argc, char* argv[])
{
int pross, rank, p_count, count = 10;
MPI_Init(&argc,&argv);
MPI_Comm_size(MPI_COMM_WORLD, &pross);
MPI_Comm_rank(MPI_COMM_WORLD, &rank);
int* num = malloc((pross-1)*sizeof(int));
// master does not send data to itself.
// only workers send data to master.
for (int i=0; i<3; ++i)
{
// to prove that no further sync is needed.
// you will get the same answer in each cycle.
p_count = 0;
if (rank == 0)
{
// this has not effect since master uses p_count for both
// send and receive buffers due to MPI_IN_PLACE.
count = 500;
MPI_Reduce(MPI_IN_PLACE, &p_count, 1, MPI_INT, MPI_SUM, 0, MPI_COMM_WORLD);
}
else
{
// for slave p_count is irrelevant.
MPI_Reduce(&count, NULL, 1, MPI_INT, MPI_SUM, 0, MPI_COMM_WORLD);
}
if (rank == 0)
{
printf("p_count = %i\n", p_count);
}
// slaves send their data to master before the cycle completes.
// no need for explicit sync such as MPI_Barrier.
// MPI_Barrier(MPI_COMM_WORLD); // no need.
}
MPI_Finalize();
}
In the code above count in slaves are reduced to p_count in master. Note the MPI_IN_PLACE and two MPI_Reduce calls. You can get the same functionality by simply setting count = 0 and calling MPI_Reduce by all ranks without MPI_IN_PLACE.
for (int i=0; i<3; ++i)
{
p_count = 0;
if (rank == 0) count = 0;
MPI_Reduce(&count, &p_count, 1, MPI_INT, MPI_SUM, 0, MPI_COMM_WORLD);
}