Resize a memory mapped file on windows without invalidating pointers

Question

I would like to resize a memory mapped file on windows, without invalidating the pointer retrieved from a previous call to MapViewOfFileEx. This way, all pointers to any file data that are stored throughout the application are not invalidated by the resize operation.

I found a solution for the problem but im not sure whether this approach is actually guaranteed to work in all cases.

This is my approach: I reserve a large memory region with VirtualAlloc:

reserved_pages_ptr = (char*)VirtualAlloc(nullptr, MAX_FILE_SIZE, MEM_RESERVE, PAGE_NOACCESS);
base_address = reserved_pages_ptr;

Every time the memory map is resized, I close the old file mapping, release the reserved pages and reserve the rest pages, that are not needed for the current size of the file:

filemapping_handle = CreateFileMappingW(...);

SYSTEM_INFO info;
GetSystemInfo(&info);
const DWORD page_size = info.dwAllocationGranularity;

const DWORD pages_needed = file_size / page_size + size_t(file_size % page_size != 0);

// release reserved pages:
VirtualFree(reserved_pages_ptr, 0, MEM_RELEASE);
// reserve rest pages:
reserved_pages_ptr = (char*)VirtualAlloc(
    base_address + pages_needed * page_size, 
    MAX_FILE_SIZE - pages_needed * page_size, 
    MEM_RESERVE, PAGE_NOACCESS
);

if(reserved_pages_ptr != base_address + pages_needed * page_size)
{
    //I hope this never happens...
}

Then i can map the view with MapViewOfFileEx:

data_ = (char*)MapViewOfFileEx(filemapping_handle, ... , base_address);

if (data_ != base_address)
{
    //I hope this also never happens...    
}

Is this approach stable enough to guarantee, that the potential problems never occur? Do I need any synchronization to avoid problems with multithreading?

EDIT: I know that the most stable approach would be to change the rest of the application to allow invalidating all the file data pointers, but this solution could be an easy approach that mirrors the behavior of mmap on Linux.

Answer 1

The solution depends on whether you use file mapping object is backed by the operating system paging file (the hFile parameter is INVALID_HANDLE_VALUE), or by some file on disk.

In this case, you use file mapping object is backed by the operating system paging file, you need use the SEC_RESERVE flag:

specifies that when a view of the file is mapped into a process address space, the entire range of pages is reserved for later use by the process rather than committed. Reserved pages can be committed to subsequent calls to the VirtualAlloc function. After the pages are committed, they cannot be freed or decommitted with the VirtualFree function.

The code can look like:

#define MAX_FILE_SIZE 0x10000000

void ExtendInMemorySection()
{
    if (HANDLE hSection = CreateFileMapping(INVALID_HANDLE_VALUE, 0, 
            PAGE_READWRITE|SEC_RESERVE, 0, MAX_FILE_SIZE, NULL))
    {
        PVOID pv = MapViewOfFile(hSection, FILE_MAP_WRITE, 0, 0, 0);

        CloseHandle(hSection);

        if (pv)
        {
            SYSTEM_INFO info;
            GetSystemInfo(&info);

            PBYTE pb = (PBYTE)pv;
            int n = MAX_FILE_SIZE / info.dwPageSize;
            do 
            {
                if (!VirtualAlloc(pb, info.dwPageSize, MEM_COMMIT, PAGE_READWRITE))
                {
                    break;
                }

                pb += info.dwPageSize;

            } while (--n);
            UnmapViewOfFile(pv);
        }
    }
}

But from SEC_RESERVE

This attribute has no effect for file mapping objects that are backed by executable image files or data files (the hfile parameter is a handle to a file).

For this (and only this) case exists undocumented API:

NTSYSCALLAPI
NTSTATUS
NTAPI
NtExtendSection(
    _In_ HANDLE SectionHandle,
    _Inout_ PLARGE_INTEGER NewSectionSize
    );

This API lets you extend section size (and backed file). Also, SectionHandle must have SECTION_EXTEND_SIZE access right in this case, but CreateFileMapping creates a section handle without this access. So we need use only NtCreateSection here, then we need use ZwMapViewOfSection api with AllocationType = MEM_RESERVE and ViewSize = MAX_FILE_SIZE - this reserve ViewSize region of memory but not commit it, but after calling NtExtendSection the valid data (commit pages) in view will be auto extended. Before win 8.1, the MapViewOfFile not such functionality for the pass MEM_RESERVE allocation type to ZwMapViewOfSection, but begin from win 8 (or 8.1) exist undocumented flag FILE_MAP_RESERVE which let do this.

In general, demonstration code can look like:

#define MAX_FILE_SIZE 0x10000000

void ExtendFileSection()
{
    HANDLE hFile = CreateFile(L"d:/ee.tmp", GENERIC_ALL, 0, 0, CREATE_ALWAYS, 0, 0);

    if (hFile != INVALID_HANDLE_VALUE)
    {
        HANDLE hSection;

        SYSTEM_INFO info;
        GetSystemInfo(&info);
        // initially only 1 page in the file
        LARGE_INTEGER SectionSize = { info.dwPageSize };

        NTSTATUS status = NtCreateSection(&hSection, 
            SECTION_EXTEND_SIZE|SECTION_MAP_READ|SECTION_MAP_WRITE, 0, 
            &SectionSize, PAGE_READWRITE, SEC_COMMIT, hFile);

        CloseHandle(hFile);

        if (0 <= status)
        {
            PVOID BaseAddress = 0;
            SIZE_T ViewSize = MAX_FILE_SIZE;

            //MapViewOfFile(hSection, FILE_MAP_WRITE|FILE_MAP_RESERVE, 0, 0, MAX_FILE_SIZE);
            status = ZwMapViewOfSection(hSection, NtCurrentProcess(), &BaseAddress, 0, 0, 0, 
                &ViewSize, ViewUnmap, MEM_RESERVE, PAGE_READWRITE);

            if (0 <= status)
            {   
                SIZE_T n = MAX_FILE_SIZE / info.dwPageSize - 1;
                do 
                {
                    SectionSize.QuadPart += info.dwPageSize;

                    if (0 > NtExtendSection(hSection, &SectionSize))
                    {
                        break;
                    }

                } while (--n);

                UnmapViewOfFile(BaseAddress);
            }
            CloseHandle(hSection);
        }
    }
}