Search code examples
unixshellloggingtee

Use tee (or equivalent) but limit max file size or rotate to new file

I would like to capture output from a UNIX process but limit max file size and/or rotate to a new file.

I have seen logrotate, but it does not work real-time. As I understand, it is a "clean-up" job that runs in parallel.

What is the right solution? I guess I will write a tiny script to do it, but I was hoping there was a simple way with existing text tools.

Imagine: 

my_program | tee --max-bytes 100000 log/my_program_log

Would give... Always writing latest log file as: log/my_program_log

Then, as it fills... renamed to log/my_program_log000001 and start a new log/my_program_log.

Solution

  • use split:

    my_program | tee >(split -d -b 100000 -)

    Or if you don't want to see the output, you can directly pipe to split:

    my_program | split -d -b 100000 -

    As for the log rotation, there's no tool in coreutils that does it automatically. You could create a symlink and periodically update it using a bash command:

    while ((1)); do ln -fns target_log_name $(ls -t | head -1); sleep 1; done