I often want large, contiguous regions of virtual address space that can grow on demand. On Windows, I do this my calling VirtualAlloc with MEM_RESERVE and a dwSize argument that I think is larger than the region is reasonably likely to ever need to grow, then committing a page at a time as needed. Not only does this defer mapping pages to physical memory until they're accessed, which committing the whole region to begin with would also do, it also defers charging the pages against the system commit limit. That way, the program doesn't limit how much memory other programs are allowed to commit for the sake of memory it isn't yet using and may never use. Basically, I want to handle my program's memory management such that it is in principle allowed to consume a large amount of memory if the user's demands require it, while at the same time allowing other programs to have that memory instead if they need it first.
Someone has already asked whether macOS has an equivalent to VirtualAlloc with MEM_RESERVE. The answers suggest that mmap with MAP_ANON | MAP_PRIVATE is roughly equivalent. What I'm wondering is, does macOS have an equivalent to the Windows commit limit, and does mmap, called with the right flags, behave like my VirtualAlloc usage in not charging against that limit?
Edit: someone else asked a similar question about Linux, for which mmap was also suggested. The answer suggests that on that platform, initially mapping the region with PROT_NONE and adding the desired privileges when the pages are needed is necessary to prevent the unused pages from counting against the commit limit. Lacking better documentation about why mapping memory in excess of the available physical memory is allowed in macOS (complete lack of commit limit? some kind of overcommit feature like Linux has?), I figure I might as well apply the same PROT_NONE trick just in case. At the very least it means I will be able to reuse the same code for Linux should I choose to support that platform as well.
I don't believe that macOS has a system commit limit. I don't find anything like that in the Mach APIs, which is the low-level VM API where I'd expect it to be. Likewise, I don't see anything like that in the output of sysctl -a, which reports many other VM details and statistics.
I can tell you for sure that I've reserved the entirety of the unused space in a 64-bit process (which has a 46-bit user address space). That is, just under 128TiB.
You can replicate that by just repeating mmap() calls with a large size until they fail, halve the size, and repeat, until your size is down to 1 page. Then, with the process paused at that point, apply vmmap -w -interleaved <pid> to it to see its allocations.