Setting maxfile on OS X/MacOS Programmatically

If I look at the maxfiles option using launchctl's limit command (on my OS X El Cap machine)

$ launchctl limit
    cpu         unlimited      unlimited      
    filesize    unlimited      unlimited      
    data        unlimited      unlimited      
    stack       8388608        67104768       
    core        0              unlimited      
    rss         unlimited      unlimited      
    memlock     unlimited      unlimited      
    maxproc     709            1064           
    maxfiles    256            unlimited

There appears to be a soft limit of 256, and a hard limit of "unlimited". I'd like to change the soft limit to be something like 2048, and leave the hard limit untouched. When I look at limit's arguments

$ launchctl help limit
Usage: launchctl limit [<limit-name> [<both-limits> | <soft-limit> <hard-limit>]

It appears I can either set both limits to the same thing, or set a value for soft and hard. However, if I attempt to set a hard limit of unlimited.

$ sudo launchctl limit maxfiles 2048 unlimited

I end up with the curious value of 10240

$ launchctl limit
    cpu         unlimited      unlimited      
    filesize    unlimited      unlimited      
    data        unlimited      unlimited      
    stack       8388608        67104768       
    core        0              unlimited      
    rss         unlimited      unlimited      
    memlock     unlimited      unlimited      
    maxproc     709            1064           
    maxfiles    2048           10240

What's going on here? Is it possible to set a value of unlimited? If not, is that a limitation of the launchctl limit command, or something on the system level? If the later, what are all those initial value of unlimited reporting? Or is this Apple being Apple?

For bonus points -- does anyone know why this limit is set so low in the first place?

Solution

From the man page for setrlimit(), the underlying system call that launchd would rely on to implement this feature:

setrlimit() now returns with errno set to EINVAL in places that historically succeeded. It no longer accepts "rlim_cur = RLIM_INFINITY" for RLIM_NOFILE. Use "rlim_cur = min(OPEN_MAX, rlim_max)".

So, while getrlimit() may (initially) report that the hard limit is RLIM_INFINITY, it is, practically, OPEN_MAX (10240) because that's simply the limit imposed by how the kernel is implemented. If you attempt to set the limit, you'll need to use the latter value, which then affects what's reported from then on.

As to why the soft limit for RLIM_NOFILE defaults to 256, it's because for the vast, vast majority of processes, this is not a practical constraint. They get along just fine without coming close to this limit. And keeping the value "so low" means that each process has a lower cost in terms of memory and each fork() has less work to do to duplicate file descriptors into new child processes, etc.

Programs which know they are likely to work with many more files can change their own limits using setrlimit().