Search code examples
ocamlmmap

Efficiently copying string to mmap'ed location


I'd like to copy a string from an OCaml program to a mmap'ed memory region (obtained via Genarray.file_map) as efficiently as possible. My objective is to allow other processes to read from this shared memory while the OCaml process is running, with minimal overhead (I do not need full concurrency features, there is only one writer and one reader).

I tried copying character per character, as in the following snippet (where I copy the first 255 characters of string s):

let fd = Unix.openfile "/tmp/foo" [Unix.O_RDWR; Unix.O_CREAT] 0o600 in
let mmap = Bigarray.Genarray.map_file fd Bigarray.Char Bigarray.C_layout true
    (Array.of_list [256])
in
let n = min (String.length s - 1) 255 in
for i = 0 to n do
  Bigarray.Genarray.set mmap [|i|] (String.get s i)
done;
Bigarray.Genarray.set mmap [|n|] (Char.chr 0)

But this is very inefficient: even with a relatively small input, it takes already 3 times longer to execute than without mmap.

Is there a better way to do it? Ideally I'd like to avoid too many dependencies, e.g. Jane Street's core.


Solution

  • The Bigarray module only provides blit between bigarrays. If the extra overhead really is due to the behavior of mmapped memory, you might try copying from the string to a bigarray first, then blit from the bigarray into the mmapped target. (To read from the mmapped array, you could do the inverse.)

    The only other thing I can think of is to code up the transfers in C (using memcpy()) and call them as external functions.