Struct having vector of structs mmapped

Question

I am trying out this scenario - write a struct (multiple instances) which has a vector of struct to mmapped file and read from mmapped file.

In the below code; when readFromMemMap() is called from the same program execution context, the read seems to be successful. But if I move readFromMemMap() to a different cpp file and run; then seg fault error occurs.

Thanks for any pointers/inputs to resolve this.

Code

#include <iostream>
#include <cstdlib>
#include <sys/types.h>
#include <sys/stat.h>
#include <unistd.h>
#include <fcntl.h>
#include <sys/mman.h>
#include <vector>

#define FILEPATH "/tmp/mmapped.bin"
#define NUMINTS  (10)

struct _3DVec
{
  int x;
  int y;
  int z;
};

struct Coords
{
  std::vector<_3DVec> coords;
};

void readFromMemMap()
{
  std::cout << "\n----------------------------------\n" << std::endl;

  int fileSize = NUMINTS * sizeof(Coords);
  std::cout << "Reading from mmapped file\n" << std::endl;

  std::cout << "FileSize = " << fileSize << "\n\tSize of struct Coords =" << sizeof(Coords) << std::endl;

  int i;
  int fd;
  Coords *map;

  fd = open(FILEPATH, O_RDONLY);
  if (fd == -1) 
  {
    std::cerr << "Error opening file for reading" << std::endl;
    exit(EXIT_FAILURE);
  }

  map = (Coords*)mmap(0, fileSize, PROT_READ, MAP_SHARED, fd, 0);

  if (map == MAP_FAILED) 
  {
   close(fd);
   std::cerr << "Error mmapping the file" << std::endl;
   exit(EXIT_FAILURE);
  }

  /* Read the file from the mmap  */
  for (i = 1; i <=3; ++i) 
  {
    std::cout << "Reading from mmap : " << i << " Coords vector size = "  << map[i].coords.size() << std::endl;

    for (_3DVec v : map[i].coords)
    {
      std::cout << " x=" <<  v.x << ", y=" <<  v.y << ", z=" << v.z << std::endl;
    }
  }

  if (munmap(map, fileSize) == -1) 
  {
    std::cerr << "Error un-mmapping the file" << std::endl;
  }
  close(fd);
 }

int main(int argc, char *argv[])
{
  int fileSize = NUMINTS * sizeof(Coords);

  std::cout << "Writing to mmapped file " << std::endl;
  std::cout << "For writing, fileSize = " << fileSize << " \n\tSize of struct Coords =" << sizeof(Coords) << std::endl;

  int i;
  int fd;
  int result;

  Coords *map;  /* mmapped array of Coords's */

  fd = open(FILEPATH, O_RDWR | O_CREAT | O_TRUNC, (mode_t)0600);
  if (fd == -1) 
  {
    std::cerr << "Error opening file for writing" << std::endl;
    exit(EXIT_FAILURE);
  }

  /* Stretch the file size to the size of the (mmapped) array of ints*/
  result = lseek(fd, fileSize-1, SEEK_SET);
  if (result == -1) 
  {
    close(fd);
    std::cerr << "Error calling lseek() to 'stretch' the file" << std::endl;
    exit(EXIT_FAILURE);
  }

  result = write(fd, "", 1);
  if (result != 1) 
  {
    close(fd);
    std::cerr << "Error writing last byte of the file" << std::endl;
    exit(EXIT_FAILURE);
  }

  /* Now the file is ready to be mmapped.*/
  map = (Coords*)mmap(0, fileSize, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
  if (map == MAP_FAILED) 
  {
    close(fd);
    std::cerr << "Error mmapping the file" << std::endl;
    exit(EXIT_FAILURE);
  }

  /* Now write to mmapped file*/

 for (int x=1; x<=3; ++x)
 {
   Coords c;

   for (i = 1; i <=4; ++i)
  {
     _3DVec v;

     v.x = i;
     v.y = i*2;
     v.z = i*3;      

     c.coords.push_back(v);
    }
    map[x] = c; 
  }

  /* Don't forget to free the mmapped memory */
  if (munmap(map, fileSize) == -1) 
  {
    std::cerr << "Error un-mmapping the file" << std::endl; 
  }

  /* Un-mmaping doesn't close the file, so we still need to do that.*/
  close(fd);

  readFromMemMap();

  return 0;
}

Compile

 g++ writeToMemMap.cpp -o writeToMemMap -std=c++11

Output

$ ./writeToMemMap
Writing to mmapped file 
For writing, fileSize = 240 
    Size of struct Coords =24

----------------------------------

Reading from mmapped file

FileSize = 240
    Size of struct Coords =24
Reading from mmap : 1 Coords vector size = 4
x=1, y=2, z=3
x=2, y=4, z=6
x=3, y=6, z=9
x=4, y=8, z=12
Reading from mmap : 2 Coords vector size = 4
x=1, y=2, z=3
x=2, y=4, z=6
x=3, y=6, z=9
x=4, y=8, z=12
Reading from mmap : 3 Coords vector size = 4
x=1, y=2, z=3
x=2, y=4, z=6
x=3, y=6, z=9
x=4, y=8, z=12

readFromMemMap() in another cpp file

 $ ./readFromMemMap
   Reading from mmap

   FileSize = 240
       Size of struct Coords =24
   Reading from mmap : 1 Coords vector size = 4
   Segmentation fault

Answer 1

Each process have their own virtual memory. One process can not access the memory of another process (except in some platform specific ways, but those cases do not apply to dynamic memory).

std::vector allocates its internal array using std::allocator by default. std::allocator allocates dynamic memory. When you write a vector to a file, that vector will refer to dynamic memory of the process that writes the vector. If you try to read that vector in another process, then that process has not allocated any dynamic memory in the virtual memory location where the original process had (unless by pure chance). Therefore using such vector has undefined behaviour.

Objects whose state is stored in dynamic memory cannot be shared between processes.

---

To share an array between processes, you need to us a flat array rather than a vector.

However, keep in mind that the size of an array that is a member of a class cannot be determined at run time. Therefore if you need that size to be dynamic, you need to create the array as a non-member.

---

Furthermore, on line

map[x] = c;

You copy assign to map[x] where you haven't actually yet created a Coord object. This has undefined behaviour since Coord is not trivially copyable. It has been purely bad luck that even the writer code didn't crash.