Cygwin - Blocking on opening a named fifo causes another thread to block on opening a regular file

Question

A blocked open of a named pipe in Cygwin causes another thread to hang when it tries to open any file, including a simple text file. The below code reproduces the issue on cygwin 3.1.6(0.340/5/3), and works fine (does not hang) on RHEL 7.

#include <unistd.h>
#include <limits.h>
#include <stdio.h>
#include <thread>
#include <sys/stat.h>
#include <fcntl.h>

void openFile() {
    int fd;

    printf("inside new thread\n");
    sleep(10); // Ensure main thread reaches call to open()
    
    printf("opening a simple file\n");
    if((fd = open("simpleFile", 0600)) == -1) { // simpleFile is a simple text file in the filesystem
        printf("failed opening a simple file\n");
    }
    printf("simple file opened successfully\n");
    close(fd);
    printf("simple file closed\n");
}

int main(int argc, char *argv[]) {
    int fd;
    char readBuffer[PIPE_BUF];
    
    printf("creating named pipe\n");
    if (mkfifo("namedPipe", 0600)) {
        printf("creating named pipe failed\n");
    }
    
    printf("creating thread\n");
    std::thread pipeCreator = std::thread(openFile);
    
    printf("opening named pipe for read\n");
    fd = open("namedPipe", O_RDONLY); // Block will only release when we echo something into namedPipe
    printf("reading from named pipe\n");
    if (read(fd, readBuffer, PIPE_BUF) == -1) {
        printf("error reading from pipe\n");
    }
    printf("read successfully from named pipe\n");
    
    pipeCreator.join();
    
    return 0;
}

Running this prints:

creating named pipe
creating thread
opening named pipe for read
inside new thread
opening a simple file

And then blocks until the other side of the namedPipe is opened. Once freed it writes the rest of the prints:

reading from named pipe
simple file opened successfully
read successfully from named pipe
simple file closed

On RHEL this prints the expected result:

creating named pipe
creating thread
opening named pipe for read
inside new thread
opening a simple file
simple file opened successfully
simple file closed

And only then the main thread hangs until something is echoed into namedPipe.

We're working on a workaround which won't block, but that involves busy waiting which isn't great. Can anyone explain this behavior?

Answer 1

On Cygwin the open syscall locks the file descriptor table for the entire duration of the syscall. Which means all open syscalls are essentially sequentialized.

See syscalls.cc/open():

extern "C" int
open (const char *unix_path, int flags, ...)
{
      . . .
      cygheap_fdnew fd;  // <-- here

And cygheap.h:

class cygheap_fdnew : public cygheap_fdmanip
{
 public:
  cygheap_fdnew (int seed_fd = -1, bool lockit = true)
  {
    if (lockit)
      cygheap->fdtab.lock ();  // <-- here
    . . .

I see no easy way around this, but I guess it should be possible to unlock the fd table once a descriptor has been created at least in case of a fifo (see fhandler_fifo), since a fifo blocks on open. You can discuss this further on cygwin-developers.