How to deal with open() returning 1

Question

I wrote a program that creates an empty text file, and prints Succeed if it succeeds.

Compile with cc main.c

#include <stdio.h>
#include <fcntl.h>
#include <unistd.h>
#include <assert.h>

int main()
{
    int fd;

    // Create empty text file
    fd = open("foo.txt", O_RDWR | O_CREAT | O_TRUNC, 0644);
    assert(fd != -1);

    fprintf(stderr, "File descriptor is %d\n", fd);

    printf("Succeed\n");
}

It works well when run with ./a.out

When run with ./a.out >&-, open() returns 1, which I understand.

But then, printf is writing into my file!

$ cat foo.txt
Succeed

I don't want this, so I wrote the following:

int main()
{
    int fd1;
    int fd2;
    int fd3;
    int fd4;

    fd1 = open("foo.txt", O_RDWR | O_CREAT | O_TRUNC, 0644);
    assert(fd1 != -1);

    fd2 = open("foo.txt", O_RDWR | O_CREAT | O_TRUNC, 0644);
    assert(fd2 != -1);

    fd3 = open("foo.txt", O_RDWR | O_CREAT | O_TRUNC, 0644);
    assert(fd3 != -1);

    fd4 = open("foo.txt", O_RDWR | O_CREAT | O_TRUNC, 0644);
    assert(fd4 != -1);

    int final_fd = fd4;
    close(fd1);
    close(fd2);
    close(fd3);

    fprintf(stderr, "File descriptor is %d\n", final_fd);

    printf("Standard output\n");
}

It works well, printf will fail and return -1 but I don't care.

I know I could optimize by checking if open() return value is greater or equal to 3 but this is just an example.

Using ./a.out >/dev/null works but is not a solution, I can't forbid the users of my program to close standard output.

Is this a correct way to deal with the problem?

Answer 1

On POSIX open will always return smallest available file descriptor. The only way you can handle this is in the beginning of your program to check the file descriptors 0 ... 2 say with isatty - if isatty returns 0 and errno is set to EBADF, the file descriptor isn't in use, and one should then open a new descriptor from /dev/null:

for (int fd = 0; fd <= 2; fd++) {
    errno = 0;
    if (! isatty(fd) && errno == EBADF) {
        open("/dev/null", O_RDWR);
    }
}

---

Notice also that the state of those streams not being open is not standards-compliant at all. C11 7.21.3p7:

7 At program startup, three text streams are predefined and need not be opened explicitly -- standard input (for reading conventional input), standard output (for writing conventional output), and standard error (for writing diagnostic output). As initially opened, the standard error stream is not fully buffered; the standard input and standard output streams are fully buffered if and only if the stream can be determined not to refer to an interactive device.

This could be considered a failure in the C startup routine - in my opinion it should open those streams to /dev/null at least

---

Though now that I tried this, I've reached the conclusion that running programs with standard streams closed is not only brain-damaged but totally brain-dead, because any fopen would also open over stdin, stdout or stderr, so I'd modify the code into:

struct stat statbuf;
for (int fd = 0; fd <= 2; fd++) {
    if (fstat(fd) == 0 && errno == EBADF) {
        fprintf(stderr, "Id10t error: closed standard IO descriptor %d\n", fd);
        abort();
    }
}