Search code examples
bashsparse-matrixio-redirectionteelogrotate

Avoid sparse files with tee and logrotate


We have our Java application running in docker. As there is the possibility that before logback kicks in something can go wrong, we log the startup output of our application to a file. In addition we use filebeat to transfer logs from docker to ELK.

So in docker's entrypoint.sh we use

    STARTUPDATE=$(date '+%Y_%m_%d_%H_%M_%S')
    exec > >(tee "/appl/log/app-$STARTUPDATE.log") 2>&1
    echo "Container startup time: $(date)"
    cat /appl/build-info.txt
    exec java -jar -Dspring.profiles.active=$SPRING_PROFILE $JAVA_ARGS $ADDITIONAL_JAVA_ARGS $1

to duplicate/redirect STDOUT/STDERR to a log file and to docker (for filebeat to pick up docker logs).

The issue now is that tee

  • does not reopen the logfile when it gets moved away by logrotate (continues to write to the deleted inode) or
  • insists to seek the old location where it left off creating a spares file (when using logrotate's copytruncate).

Is there any suggestions of a different way to do this? Any (standard unix tool) tee replacement that does reopen and/or not seek?

Or is there some bash magic to redirect/duplicate one stream to multiple endpoints? (I read somewhere that zsh can do this like cat file > file1 > file2 - not sure if any of the "fileX" could be STDOUT again though)


Solution

  • After no useful findings and also no response from the GNU coreutils devs on their mailing list on tee, I replaced tee with good old perl one liner:

    exec > >( perl -pe 'open($fh, ">>", "'$LOGFILE'"); print $fh $_; close($fh)' ) 2>&1
    

    This seems to work stable under the given load of STDOUT.