Search code examples
cc99variable-length-arraymmu

How many memory pages do C compilers on desktop OSes use to detect stack overflows?


This question is related to but different from this one about variable length arrays in C99.

The answers point out that one danger with allocating variable length arrays (or just large arrays of a fixed size) in the stack is that the allocation may fail silently, as opposed to, say, calling malloc, which explicitly tells the caller whether allocation succeeded.

Modern non-embedded compilation platforms use an invalid memory zone to detect some stack overflows at no additional cost (the checks are only the checks already made for free by the MMU). This doesn't protect at 100% from the above problem because a very large local array may cause the stack pointer to jump over the invalid area.

Does any one know how many pages are typically allocated for this detection? I guess it would be at least 4KiB, but it could be more. Is that a choice made by the compiler or the OS, and in either case, is there a way to change it?


Solution

  • On Windows, it's one 4KB page (at least on x86): See Description of the stack checking for Windows NT-based applications.

    This automatic growth method uses a guard page, a reserved, uncommitted, memory page that is contiguous with the committed portion of memory. When the application touches the guard page, the operating system commits that page and the next uncommitted page becomes the new guard page. Automatic stack growth works only for the guard page and stack memory must grow in 4K, or one page, increments. If the application touches another reserved but uncommitted page of stack memory before it touches the guard page, a normal page fault exception occurs and unpredictable behavior can result.

    ...

    To prevent the fault, the compiler calls the __chkstk() function each time the local allocation exceeds 4K. The Windows NT __chkstk() function does not explicitly check for stack overflow as the MS-DOS version does. It simply touches memory addresses every 4K from the current stack pointer location to the requested allocation. This triggers the guard pages in the proper sequence and commits additional memory to the stack as required.

    For GCC, GCC Stack checking

    I'm not sure how/if C99's VLA's would change the WinNT behaviour.