Less gets keyboard input from stderr?

I'm taking a look at the code to the 'less' utility, specifically how it gets keyboard input. Interestingly, on line 80 of ttyin.c, it sets the file descriptor to read from:

     /*
      * Try /dev/tty.
      * If that doesn't work, use file descriptor 2,
      * which in Unix is usually attached to the screen,
      * but also usually lets you read from the keyboard.
      */
  #if OS2
      /* The __open() system call translates "/dev/tty" to "con". */
      tty = __open("/dev/tty", OPEN_READ);
  #else
      tty = open("/dev/tty", OPEN_READ);
  #endif
      if (tty < 0)
          tty = 2;

Isn't file descriptor 2 stderr? If so, WTH?! I thought keyboard input was sent through stdin.

Interestingly, even if you do ls -l * | less, after the file finishes loading, you can still use the keyboard to scroll up and down, but if you do ls -l * | vi, then vi will yell at you because it doesn't read from stdin. What's the big idea? How did I end up in this strange new land where stderr is both a way to report errors to the screen and read from the keyboard? I don't think I'm in Kansas anymore...

Solution

$ ls -l /dev/fd/
lrwx------ 1 me me 64 2009-09-17 16:52 0 -> /dev/pts/4
lrwx------ 1 me me 64 2009-09-17 16:52 1 -> /dev/pts/4
lrwx------ 1 me me 64 2009-09-17 16:52 2 -> /dev/pts/4

When logged in at an interative terminal, all three standard file descriptors point to the same thing: your TTY (or pseudo-TTY).

$ ls -fl /dev/std{in,out,err}
lrwxrwxrwx 1 root root 4 2009-09-13 01:57 /dev/stdin -> fd/0
lrwxrwxrwx 1 root root 4 2009-09-13 01:57 /dev/stdout -> fd/1
lrwxrwxrwx 1 root root 4 2009-09-13 01:57 /dev/stderr -> fd/2

By convention, we read from 0 and write to 1 and 2. However, nothing prevents us from doing otherwise.

When your shell runs ls -l * | less, it creates a pipe from ls's file descriptor 1 to less's file descriptor 0. Obviously, less can no longer read the user's keyboard input from file descriptor 0 – it tries to get the TTY back however it can.

If less has not been detached from the terminal, open("/dev/tty") will give it the TTY.

However, in case that fails... what can you do? less makes one last attempt at getting the TTY, assuming that file descriptor 2 is attached to the same thing that file descriptor 0 would be attached to, if it weren't redirected.

This is not failproof:

$ ls -l * | setsid less 2>/dev/null

Here, less is given its own session (so it is no longer a part of the terminal's active process group, causing open("/dev/tty") to fail), and its file descriptor 2 has been changed – now less exits immediately, because it is outputting to a TTY yet it fails to get any user input.