Search code examples
djangobashcrontailtee

Cronned Django command output not posted in tee from bash script

I'm trying to have a bash script that controled by cron to be ran every day and I would like to grep some lines from the python (Django) output and post it with slacktee to my slack channel. But I am only catching some warnings from the script, not my own prints (something to do with std::out and std::err)? But I can't seem to be able to debug it. 

#!/bin/bash

printf "\nStarting the products update command\n"
DATE=`date +%Y-%m-%d`
source mypath/bin/activate

cd some/path/_production/_server

./manage.py nn_products_update > logs/product_updates.log

tail --lines=1000 logs/product_updates.log | grep INFO | grep $DATE

So for each day, I'm trying to grep messages like these: 

[INFO][NEURAL_RECO_UPDATE][2017-08-28 22:15:04] Products update...

But it doesn't get printed in the tee channel. Moreover, the file gets overwritten everyday and not appended - how to change that, please? The tail command works ok when ran in the shell by itself. How is it possible? (sorry for asking two, but I believe they are somehow related, just cant find an answer)

This is just in case the cron entry. 

20 20 * * * /bin/bash /path/to/server/_production/bin/runReco.sh 2>&1 | slacktee.sh -c 'my_watch'

Many thanks

EDIT:

output when using grep -e INFO -e $DATE

grep: [INFO][NEURAL_RECO_UPDATE][2017-08-29: No such file or directory
grep: 07:36:56]: No such file or directory
grep: No: No such file or directory
grep: new: No such file or directory
grep: active: No such file or directory
grep: products: No such file or directory
grep: to: No such file or directory
grep: calculate.: No such file or directory
grep: Terminating...: No such file or directory
Solution

  • Using:

    #!/bin/bash
set -euo pipefile

    this will provide better debug output from your script See bash strict mode article for a full explanation of the settings.

    The file gets overwritten because you are using a single > (redirection) rather than >> which appends redirected output.

    To help debug further it would probably make life easier if you put

    2>&1 | slacktee.sh -c 'my_watch'

    In your runReco.sh as in:

    tail --lines=1000 logs/product_updates.log | grep INFO | grep $DATE 2>&1 | slacktee.sh -c 'my_watch'

    Although chaining lots of commands together in shell scripts makes them harder to debug.

    Hence breaking up the tail line:

    TMPFILE=`tail --lines=1000 logs/product_updates.log`
grep -e INFO -e $DATE $TMPFILE 2>&1 | slacktee.sh -c 'my_watch'