Search code examples
clinuxmmap

mmap file with larger fixed length with zero padding?


I want to use mmap() to read a file with fixed length (eg. 64MB), but there also some files < 64MB.

I mmap this files (<64MB, eg.30MB) with length = 64MB, when read file data beyond file size(30MB - 64MB), the program got a bus-error.

I want mmap these files with fixed length, and read 0x00 when pointer beyond file size. How to do that?

One method I can think is ftruncate file first, and ftruncate back to ori size, but I don't think this method is perfect.


Solution

  • This is one of the few reasonable use cases for MAP_FIXED, to remap part of an existing mapping to use a new backing file.

    A simple solution here is to unconditionally mmap 64 MB of anonymous memory (or explicitly mmap /dev/zero), without MAP_FIXED and store the resulting pointer.

    Next, mmap 64 MB or your actual file size (whichever is less) of your actual file, passing in the result of the anonymous/zero mmap and passing the MAP_FIXED flag. The pages corresponding to your file will no longer be anonymous/zero mapped, and instead will be backed by your file's data; the remaining pages will be backed by the anonymous/zero pages.

    When you're done, a single munmap call will unmap all 64 MB at once (you don't need to separately unmap the real file pages and the zero backed pages).

    Extremely simple example (no error checking, please add it yourself):

    // Reserve 64 MB of contiguous addresses; anonymous mappings are always zero backed
    void *mapping = mmap(NULL, 64 * 1024 * 1024, PROT_READ, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
    
    // Open file and check size
    struct stat sb;
    int fd = open(myfilename, O_RDONLY);
    fstat(fd, &sb);
    // Use smaller of file size or 64 MB
    size_t filemapsize = sb.st_size > 64 * 1024 * 1024 ? 64 * 1024 * 1024 : sb.st_size;
    // Remap up to 64 MB of pages, replacing some or all of original anonymous pages
    mapping = mmap(mapping, filemapsize, PROT_READ, MAP_SHARED | MAP_FIXED, fd, 0);
    close(fd);
    
    // ... do stuff with mapping ...
    munmap(mapping, 64 * 1024 * 1024);