How would I define a new type of java.nio.channels.SelectableChannel (say for serial ports)?
My understanding is that the java implementation is based on the unix select() c function (I seem to remember that the Windows implementation was slightly different)
Depending on OS (and JVM args!) different native OS functions are called, but what they have in common is that it's native code - the basic functionality is not implemented in Java.
If you want to make a lib that access the select() (or similar) of the underlying OS (which does indeed rely on filehandles), I think you are pretty much forced to use JNI. I don't believe there are any ways around it.
The Selector/SelectableChannel in Java is really an anemic subset of what select() can do.