Search code examples
macossetrlimit

What does _RLIMIT_POSIX_FLAG do on macOS?

In sys/resource.h header on macOS, you will find this definition:

#define _RLIMIT_POSIX_FLAG      0x1000  /* Set bit for strict POSIX */

What does _RLIMIT_POSIX_FLAG actually do?

Solution

  • _RLIMIT_POSIX_FLAG was introduced in XNU 1228 (macOS 10.5), which was the first release of macOS certified for UNIX 03 compatibility. (These old versions were branded Mac OS X, so calling them by the contemporary term macOS is maybe somewhat anachronistic, but I will do it anyway.)

    As part of UNIX 03 compatibility, they had to make some changes to the setrlimit system call to pass the UNIX 03 test suite. The problem was that changing the behaviour of the system call would break older software relying on the pre-10.5 behaviour. So they modified the setrlimit syscall wrapper to add the _RLIMIT_POSIX_FLAG to all setrlimit system calls. Hence new software using the new version of the wrapper would pass that flag to get the new behaviour, whereas legacy software using the pre-UNIX03 wrapper would not pass the flag and get the original behaviour.

    In bsd/kern/kern_resource.c, the _RLIMIT_POSIX_FLAG is detected and used to trigger different behaviour in certain cases:

    • RLIMIT_STACK: POSIX requires returning EINVAL if an illegal stack limit is detected; without _RLIMIT_POSIX_FLAG, setrlimit clips to the maximum valid limit instead. There is a comment referencing "poorly written installers" which passed an illegal stack limit.
    • RLIMIT_NOFILE: similarly, pre-POSIX behaviour is to clamp request for limit exceeding system-wide limit to the system-wide limit, POSIX behaviour is to return EINVAL instead

    They similarly set the _RLIMIT_POSIX_FLAG bit in the wrapper for getrlimit, but while the getrlimit accepts the flag, it doesn't do anything in response. A comment in the source reads "Take out flag now in case we need to use it to trigger variant behaviour later", implying that they added it just in case they needed to change the getrlimit behaviour for POSIX compliance but in the end decided no changes were necessary, but still kept the flag in getrlimit anyway.

    In XNU 8019 (macOS 12), they removed the code from setrlimit that supports the legacy pre-10.5 setrlimit behaviour – compare dosetrlimit in 8019.41.5 to dosetrlimit in 7195.50.7.100.1.

    So since macOS 12, _RLIMIT_POSIX_FLAG no longer does anything at all. It is a useless bit always set in the getrlimit and setrlimit system calls, but the calls behave identically whether or not it is set. I assume Apple removed the legacy pre-10.5 behaviour in macOS 12 because 10.5 was the first release to support 64-bit apps, so all 10.4 and earlier apps were 32-bit only, and 32-bit app support was removed in macOS 10.15, so apps that rely on the pre-10.5 setrlimit behaviour cannot run any more. But it seems like it took them until macOS 12 to realise this legacy support code was now pointless and actually remove it (or maybe they realised it earlier but they just didn't get around to it until then.)