How can we limit file descriptor to always be less than 1024?

Question

There is limitation in select system call that it will not work beyond 1024. This is what document says

WARNING: select() can monitor only file descriptors numbers that
       are less than FD_SETSIZE (1024)—an unreasonably low limit for
       many modern applications—and this limitation will not change.
       All modern applications should instead use poll(2) or epoll(7),
       which do not suffer this limitation.

we have experienced this behaviour only in AWS EC2 instance. Outside of AWS fd is always < 1024

Is there a setting in Linux system (Suse Linux 15 SP2) that will always create fds < 1024?

Answer 1

You don't want to do this. Linux always uses the lowest possible file descriptor. So if you get file descriptor 1024 then it means file descriptors 0 up to 1023 were all used already.

If you make Linux only use file descriptors 0-1023, then your program still won't work, because instead of getting file descriptor 1024, you'll get an error saying there aren't any more file descriptors it can use.

You should:

1. Make sure your program closes file descriptors when it's done with them.
   Maybe the reason descriptors 0-1023 were all used is because you forgot to close them. So many sure your program closes them.
   If they were real file descriptors that your program was actually using, not just ones you forgot to close, then continue to step 2...
2. Use poll instead of select, like the document says. It is not a difficult change.
3. Consider using epoll, which is more efficient than select when you are polling a lot of file descriptors at the same time. However, it is much more complicated.