How to properly write C structure to file on disk, so it was possible to use mmap on it?

Question

I use following C structure in memory:

typedef struct MyStructHdr
{

    char        text[4];
    int         version_num;

    uint64 init_value;

    uint64 entries[];

} MyStructHdr;

typedef MyStructHdr *MyStruct;

Field entries[] is a pointer to some flexible array. Type uint64 is a custom portable application specific type, which add uint64_t support on 32-bit OS.

I have to properly write this structure to file, so that I was able to use mmap() on it later (on the same platform/OS):

map = (MyStruct) mmap(NULL, MyStructActualSize,
                      PROT_READ | PROT_WRITE, MAP_SHARED,
                      mystruct_fd, 0);

What I do now? I simply write MyStruct fields one by one (and entries[] by chunks via buffer) using write(). At the end CRC32 checksum is written.

Everything works just fine on all available to me 64-bit systems. It seems that first 4 chars + 32-bit int are aligned into the single 64-bit chunk and uint64 simply expands into uint64_t, so after write everything is mmap'ed correctly.

However, I am afraid that on 32-bit system or some specific OS/architecture, where different alignment rules are applied and there is no uint64_t and uint64 expands into something like:

{
    int val1;
    unsigned long int val2;
}

I will get incorrect mmap'ing after write.

What is a portable way to write such a structure to file and use mmap after that?

P.S. Actually, this is all about PostgreSQL extension and uint64 here is pg_atomic_uint64, but I think that question is more general.

Answer 1

You shouldn't write the members one by one, because that won't account for padding between members. Write the whole thing all at once:

write(fd, MyStruct, sizeof(MyStructHdr) + entry_count * sizeof(uint64));

where entry_count is the number of elements in the flexible array member.

If you need to write each member separately for atomicity, you can use the offsetof macro to get the size including padding.

write(fd, &MyStruct->text, offsetof(MyStructHdr, version_num));
write(fd, &Mystruct->version_num, offsetof(MyStructHdr, init_value) - offsetof(MyStructHdr, version_num));
write(fd, &MyStruct->init_value, offsetof(MyStructHdr, entries) - offsetof(MyStructHdr, init_value));

then write the MyStruct->entries array in chunks. You don't need to worry about padding there, because sizeof on an array element includes the padding between elements (this ensures that sizeof array == element_count * sizeof array[0]);