I want to be able to output to both stdout(&stderr) AND to another file (log file) without worrying that the log file would get corrupted output for the case when shell is redirecting stdout(and/or stderr) to the same log file.
For my particular case, I've tried checking the stat bits(man 2 stat) for stdout and for the log file, in order to determine that they point to the same device and inode, in which case don't fopen() the log file but instead fopen() the stdout file, for writing to the log file.
Exactly what I mean(.c code): https://github.com/libcheck/check/issues/188#issuecomment-492852881
This works as a workaround.
Here's a .c code example:
#include <stdio.h>
int main() {
FILE *f=NULL;
f = fopen("/tmp/a_out_.log", "w");
if (NULL == f) {
fprintf(stderr,"oopsie\n");
} else {
fprintf(stdout, "Something");
fprintf(f," messy ");
fprintf(f," jessy\n");
fprintf(stdout, " or another\n");
fprintf(f,"More stuff\n");
fclose(f);
}
}
Run like this(from bash) to see the overwritten output:
$ gcc a.c && { ./a.out >/tmp/a_out_.log ; cat /tmp/a_out_.log ; }
Something or another
uff
I've simplified the .c code and reduced it to a bash lines, but the functionality (ie. garbled output) is illustrated just the same:
All of these show correct output:
(echo "1 2 3 4 5 6 7 8 9 10" ; echo "blah" >&2)
(echo "1 2 3 4 5 6 7 8 9 10" ; echo "blah" >&2) 1>/tmp/good 2>&1 ; cat /tmp/good
(echo "1 2 3 4 5 6 7 8 9 10" ; echo "blah" >&2) 1>/dev/stdout 2>/dev/stdout
(echo "1 2 3 4 5 6 7 8 9 10" ; echo "blah" >&2) 1>/proc/self/fd/1 2>/proc/self/fd/1
(echo "1 2 3 4 5 6 7 8 9 10" ; echo "blah" >&2) 1>/proc/self/fd/2 2>/proc/self/fd/2
output is:
1 2 3 4 5 6 7 8 9 10
blah
But, the following one shows overwritten output:
(echo "1 2 3 4 5 6 7 8 9 10" ; echo "blah" >&2) 1>/tmp/bad 2>/tmp/bad; cat /tmp/bad
(the corrupted)output looks like:
blah
4 5 6 7 8 9 10
Real world example of where this is happening(with reproduction steps even): https://github.com/libcheck/check/issues/188
Why are parts of the output overwritten when redirected to the same file but not when on terminal?
Because you have the file open twice, separately, on a regular file, in ordinary write mode. Each open file description for that file has its own sense of the current file position, and that's where the data it writes go. Each file position advances only in accordance with the data written via the corresponding open file description. Whichever data happens to be written second at a given position replaces what was written first.
This does not happen for a terminal, because terminals are not seekable. It's as if they are always open in append mode. Opening your log file in append mode would provide half of a solution, and would be a good idea in any case:
#include <stdio.h>
int main() {
FILE *f=NULL;
f = fopen("/tmp/a_out_.log", "a"); // <-- here is the change
if (NULL == f) {
fprintf(stderr,"oopsie\n");
} else {
fprintf(stdout, "Something");
fprintf(f," messy ");
fprintf(f," jessy\n");
fprintf(stdout, " or another\n");
fprintf(f,"More stuff\n");
fclose(f);
}
}
That way, writes to file f always go to the current end of file, regardless of what may have been done to the file by other means. If you want an existing log file to be truncated, however, then you'll have to do that yourself, unlike when you open in write ("w") mode.
As I said, however, although opening in append mode is probably a good idea in this case regardless, it's only half a solution. If the standard output is open on the same file, separately, in regular write mode, then writes from that direction still can and will overwrite other output. Frankly, I'd be inclined to say that that should not be your program's concern. If the user really wants to redirect the program's console output to its log file, then it is within their power to do so in append mode, by using the >> redirection operator instead of >. That would constitute the complement of the above half solution. If they use a > redirection instead then that's on them, and I would not go to extraordinary measures to detect or accommodate it.