Search code examples
clinuxfilemallocmmap

What should use mmap, malloc or File I/O


Background Our kernel level program invokes a process in user space for making some decisions on the basis of values in a file. The user space program is a short lived process that compares value passed by the kernel with the file contents. At a time usually many instances of the user space program can be invoked. The file has less than one thousand lines.

Question What is the preferred way to read the a small file that is shared among short lived many processes? Currently We are using File I/O (fopen, fread)

Note The question When should I use mmap for file access? discusses very nicely but there is no discussion for the case of short lived many processes


Solution

  • What is the preferred way to read a small file that is shared among short lived many processes?

    getline() or fread() using standard POSIX I/O from <stdio.h>, or low-level <unistd.h> open() and read() to a sufficiently large buffer (with sufficiently aggressive growth policy); depending on how the read data is parsed/interpreted.

    You don't use memory mapping for reading a file once; it is just not as efficient as read()/fread(), due to the mapping overhead.

    Note that if the file contains many numbers, the actual bottleneck is the string-to-integer and string-to-floating-point conversions (strtol(), strtod(), sscanf(), etc.), because if accessed often enough the file contents will stay in the page cache. The standard implementations of string conversion functions are designed for correctness, not for efficiency.

    Our kernel level program invokes a process in user space for making some decisions on the basis of values in a file.

    Seems very inefficient to me. Personally, I'd keep the "file" in-kernel, as a structure, and only expose an userspace interface, probably a character device, to modify its contents.

    That way you only incur a context switch whenever the "file" is changed by an userspace process, and kernel-space stuff can simply examine the contents of the structure directly, in native format, with no overhead.

    This is what e.g. netfilter (built-in firewall) and other existing stuff do.