Search code examples
mpibroadcast

broadcasting structs without knowing attributes


I'm trying to use MPI_Bcast to share an instance of cudaIpcMemHandler_t, but I cannot figure out how to create the corresponding MPI_Datatype needed for Bcast. I do not know the underlying structure of the cuda type, hence methods like this one don't seem to work. Am I missing something ?


Solution

  • Following up from the useful comment , I have a solution that seems to work, though its only been tested in a toy program:

    //  Broadcast the memory handle
    
        cudaIpcMemHandler_t  memHandler[1];
        if (rank==0){
            // set the handler content, e.g. call cudpaIpcGetMemHandle
        }
        MPI_Barrier(MPI_COMM_WORLD);
        
        // share the size of the handler with other objects
        int hand_size[1];
        if (rank==0)
            hand_size[0]= sizeof(memHand[0]);
        MPI_Bcast(&hand_size[0], 1, MPI_INT, 0, MPI_COMM_WORLD);
    
        // broadcase the handler as byte array
        char memHand_C[hand_size[0]];
        if (rank==0)
            memcpy(&memHand_C, &memHand[0], hand_size[0]);
        MPI_Bcast(&memHand_C, hand_size[0], MPI_BYTE, 0, MPI_COMM_WORLD);
        if (rank >0)
            memcpy(&memHand[0], &memHand_C, hand_size[0]);