Search code examples
clinuxprocessshared-memorymmap

Using mmap with hint address is the address of a global variable and MAP_FIXED


I have 2 or more processes accessing to a shared memory. I want to create a global variable in each process and then mapping the address of this variable to the shared memory using mmap API with MAP_FIXED flag. So that, when reading/ writing to this shared memory, I just need to access to the global variable ( the same way as we share global variable between threads, but here I would like to shared global variable between processes).

I define the global variable in each process as below:

typedef struct     // This struct define the shared memory area
{
   int data1;
   int data2;
   // ...
} SharedMemory;

// the following attribute (supported by GCC) make the start address of the variable aligned to 4KB (PAGE_SIZE)
 __attribute__((aligned(4096))) SharedMemory gstSharedMemory; // shared global variable
 int giOtherVar = 10;                                              // Another normal global variable

Then using mmap to map the shared memory to this global variable:

void* lpShmAddr = mmap(&gstSharedMemory,
                        sizeof(gstSharedMemory),
                        PROT_READ | PROT_WRITE,
                        MAP_SHARED | MAP_FIXED,
                        iFd,                        // File descriptor to the shared memory
                        0);

However, if sizeof(gstSharedMemory) is not multiple of PAGE_SIZE (4kb), and because the OS will always round up the map size to multiple of page size, all the bytes in the rounded-up region are initialized to 0. And it may cause the data of other global variable (for example: giOtherVar) to become Zero if their address is within this rounded-up region.

To overcome this situation, I use a byte array to backup the rounded-up region and recover it as below:

unsigned char byBkupShm[PAGE_SIZE]    =  { 0 } ;    
memcpy(&gbyBkupShm[0], 
      ((unsigned char*)&gstSharedMemory+ sizeof(gstSharedMemory)), 
       PAGE_SIZE - (sizeof(gstSharedMemory)% PAGE_SIZE));
void* lpShmAddr = mmap(&gstSharedMemory,
                        sizeof(gstSharedMemory),
                        PROT_READ | PROT_WRITE,
                        MAP_SHARED | MAP_FIXED,
                        iFd,                        // File descriptor to the shared memory
                        0);
 memcpy( ((unsigned char*)&gstSharedMemory+ sizeof(gstSharedMemory)), 
          &byBkupShm[0],
          PAGE_SIZE - (sizeof(gstSharedMemory)% PAGE_SIZE));

And finally, I access to shared memory like this:

// Write to shared memory:
gstSharedMemory.data1 = 5;

// Read from shared memory;
printf("%d", gstSharedMemory.data1);

My question is: Is there any potential problem with this implementation?

Editted: Thank to @None and his idea, I define a macro as below to make my struct aligned and rounded up to PAGE_SIZE, but at the same time, still provide the actual size of the struct if I need:

#define PAGE_SIZE       (4 * 1024)                                  // Page Size: 4KB
#define SHM_REG          __attribute__((aligned(PAGE_SIZE)))        // Aligned to 4KB boundary

#define DEFINE_SHM( structName_, shmSizeVar_, structContent_)       \
typedef struct SHM_REG structContent_ structName_;                  \
int shmSizeVar_ = sizeof(struct structContent_);  



// Using

DEFINE_SHM(
    MySharedMemory,                 // Struct Name of shared memory
    giSizeOfMySharedMemory,         // Global Variable 
    {
        int a;
        int b;
        char c;
    }
);
    
printf("Rounded Size: %d\n", sizeof(MySharedMemory));  // = 4096
printf("Acutal Size: %d\n", giSizeOfMySharedMemory);   // = 12 

Solution

  • Make sure the shared memory structure is both aligned to and sized as a multiple of page size:

    #include <stdlib.h>
    
    #ifndef  PAGE_SIZE
    #define  PAGE_SIZE  4096
    #endif
    
    typedef struct __attribute__((aligned (PAGE_SIZE))) {
        /*
         * All shared memory members
        */
    } SharedMemory;
    

    At run time, before mapping the shared memory, verify it first:

    SharedMemory  blob;
    
        if (PAGE_SIZE != sysconf(_SC_PAGESIZE)) {
            ABORT("program compiled for a different page size");
        } else
        if (sizeof blob % PAGE_SIZE) {
            ABORT("blob is not sized properly");
        } else
        if ((uintptr_t)(&blob) % PAGE_SIZE) {
            ABORT("blob is not aligned properly");
        } else
        if (MAP_FAILED == mmap(...)) {
            ABORT("could not map shared memory over blob");
        }
    

    While this is a hack, at least this way it is safe, in Linux.