I want to know how many channels can a selector can handle at maximum? I'm familiar with common I/O algorithms, but I don't know in NIO
whether I can use a selector for handling (for example) 10000 sockets. Should use a number of threads and have a selector in each for handling a specific number of sockets?
While I found this, it didn't address this specific question.
There is no limit other than the number of socket descriptors. Some platforms have underlying limits, but NIO works around them with multiple OS selectors per Selector.
There was a limit up to about Java 1.4.1 but it's long gone.
Should use a number of threads and have a selector in each for handling a specific number of sockets?
It's possible, but I don't really see why you should. Maybe the peers might get more regular service that way, it depends what your code has to do with every request.