Search code examples
multiprocessingmpihpcmpfr

mpfr_t in custom MPI structs


I am working on converting my program to use arbitrary precision and thus I started utilizing MPFR. I had a custom MPI struct containing my data in my original code. Now that I have converted my data to mpfr_t, I am unsure how to make this struct in MPI.

typedef struct job_s {
    int f;
    mpfr_t l,t,r,b;
} jobs;

MPI_Datatype MPI_JOBS_TYPE;
MPI_Type_create_struct(
    5,
    (int []) {1, sizeof(mpfr_t), sizeof(mpfr_t), sizeof(mpfr_t), sizeof(mpfr_t)},
    (MPI_Datatype []) {offsetof(jobs,f), offsetof(jobs,l), offsetof(jobs,t), offsetof(jobs,r), offsetof(jobs, b)},
    (MPI_Datatype []) {MPI_INT, MPI_BYTE, MPI_BYTE, MPI_BYTE, MPI_BYTE},
    &MPI_JOBS_TYPE
);

MPI_Type_commit(&MPI_JOBS_TYPE);

This struct then gets sent later with

jobs send;
/* set the objects of the struct to whatever using MPFR */
MPI_Send(&send, 1, MPI_JOBS_TYPE, reqRank, 0, MPI_COMM_WORLD);

I'm not sure where to go from here. I understand this is very wrong as there is no way for MPI to determine the size of the struct as I haven't even set the precision yet (does it even work like that?). I would appreciate any help! Thank you.

EDIT: I'm so close but something is still wrong with exporting, sending, receiving, or importing the mpfr_t vars. I can successfully pack f, send, receive, and unpack!

char * mpfr_buf;
size_t mpfr_buf_size;
char buf [3000];
pos = 0;
MPI_Pack(&f, 1, MPI_INT, buf, 3000, &pos, MPI_COMM_WORLD);

stream = open_memstream(&mpfr_buf, &mpfr_buf_size);
for(i = 0; i < 4; ++i)
    mpfr_fpif_export(stream, bounds[i]);
MPI_Pack(stream, 3000-sizeof(int), MPI_BYTE, buf, 3000, &pos, MPI_COMM_WORLD);

print_bounds(bounds, "s");

MPI_Send(buf, pos, MPI_PACKED, reqRank, 0, MPI_COMM_WORLD);

fclose(stream);
f += 1;

and on the receiving side

pos = 0;
char buf [3000];
MPI_Recv(buf, 3000, MPI_PACKED, PARENT, 0, MPI_COMM_WORLD, &status);
int msgSize = 0;
MPI_Get_count(&status, MPI_BYTE, &msgSize);
printf("msg size %d\n", msgSize);
MPI_Unpack(buf, 3000, &pos, &f, 1, MPI_INT, MPI_COMM_WORLD);
printf("Worker %d recived frame %d\n", rank, f);

char streambuf [3000];
MPI_Unpack(buf, 3000, &pos, streambuf, msgSize - pos, MPI_BYTE, MPI_COMM_WORLD);
stream = fmemopen(streambuf, 3000, "r");
for(i = 0; i < 4; ++i){
    mpfr_fpif_import(bounds[i], stream);
}
fclose(stream);

I get garbage for the first 3 I import and for the last one I SIGBART with the error get_str.c:157: MPFR assertion failed: size_s1 >= m. Once again any and all help is welcome! Thanks.


Solution

  • Unfortuantely, it is not as easy to send MPFR numbers around as MPI datatypes without breaking source or binary compatibility. The problem is that mpfr_t is a structure with an element of pointer type, therefore you cannot simply serialise it as a pure sequence of bytes. You have to register an MPI datatype that corresponds to MPFR's internal structure and use it when defining the MPI datatype for your own structure.

    But you cannot create just one MPI type that corresponds to any instance of mpfr_t. MPI does not support pointer chasing and given that the pointer element in mpfr_t can point to locations at arbitrary and most likely different in each instance offset (MPI's structured type creation works with offsets from the buffer address), you need to create a separate MPI datatype for each and every variable of type mpfr_t. This is possible, but very far from optimal because:

    • you are breaking the data encapsulation rule by peeking inside mpfr_t
    • your code will become unportable to hybrid architectures
    • it is a lot of work and really hard to debug

    The portable solution is to use the MPFR functions for Input and Output, and in particular the two experimental functions that read and write MPFR instances from and to C FILE streams. What you need to do is create a memory stream with open_memstream and use mpfr_fpif_export to write in it the portable representations of l, t, r, and b, then pack the resultant bytes in an MPI buffer using MPI_Pack. You'll need to manually pack f as well. On the receiver side, unpack f with MPI_Unpack, then unpack the byte array and open it as a memory stream with fmemopen in read mode, then use mpfr_fpif_import to read the four values out of it. As the representations may be of varying size, you should use MPI_Probe on the receiver side to first obtain the actual size of the message and to then allocate a large enough buffer for MPI_Recv.

    If the experimental nature of the FPIF export and import is somehow throwing you off, then use the string input/output.