mmap buffer to file handle

Question

I need to "fake" a file descriptor (that supports fstat) and I do it like this.

func ScanBytes(b []byte) error {
  size := C.size_t(len(b))
  path := C.CString("/bytes")
  fd := C.shm_open(path, C.O_RDWR|C.O_CREAT, C.mode_t(0600))
  defer C.shm_unlink(path)
  defer C.close(fd)

  res := C.ftruncate(fd, C.__off_t(size))
  if res != 0 {
    return fmt.Errorf("could not allocate shared memory region (%d)", res)
  }

  var addr = unsafe.Pointer(&b[0])
  C.mmap(addr, size, C.PROT_READ|C.PROT_WRITE, C.MAP_SHARED|C.MAP_FIXED, fd, 0)
  defer C.munmap(addr, size)

  // _, err := syscall.Write(int(fd), b)

  return doSomethingWith(fd)
}

You see where the write to the file handle is commented out.

If I don't write the buffer to the allocated region, it is empty. I was hoping mmap in conjunction with MAP_FIXED would use the address of the provided buffer, thus mapping the contents to that region.

I guess the write call copies, thus doubling the memory usage. Do I really have to write?

Answer 1

It sounds like what you want is to be able to access an existing memory region via a file descriptor without copying.

The problem sounds very much like this previous stackoverflow question about mmap.

There isn't much to add to that answer there. It isn't possible to do this without a copy, given that you don't control the allocation of b. Accepting that a copy is required, your shm solution is good. mmap is not needed.

From the man page for mmap, the definition of MAP_FIXED:

Do not permit the system to select a different address than the one specified. If the specified address cannot be used, mmap() will fail. If MAP_FIXED is specified, addr must be a multiple of the pagesize. If a MAP_FIXED request is successful, the mapping established by mmap() replaces any previous mappings for the process' pages in the range from addr to addr + len. Use of this option is discouraged.

It is quite possible that unsafe.Pointer(&b[0]) is not a multiple of the page size, especially if the underlying array is small. mmap would fail in that case. As always, check the return value.

If mmap does succeed, the mapping replaces any previous mapping, meaning your data is now gone. Reading the slice would give you zeroed bytes.