I have a nightly cronjob that runs a python script. The job sends all of the outputs to a log wherein each line is timestamped to the microsecond.
Part of that script includes a loop with a 5-second pause between each iteration. When I was troubleshooting some issues, I noticed that the cronjob timestamp did not reflect that 5-second pause. So I then included some python code to print its own timestamp, and the 5-second pause is clearly shown.
So, my question is:
Is my timestamp format correct for the cronjob log?
If so - why is the 5-second pause not showing?
If not - what is the correct format?
cronjob:
python myscript.py | ts '[\%Y-\%m-\%d \%H:\%M:\%.S]' >> cronjob.log 2>&1
output:
[2024-09-05 00:23:24.226220] > Doing something (time = 2024-09-05 00:22:19.738), attempt #1...
[2024-09-05 00:23:24.226234] > Doing something (time = 2024-09-05 00:22:24.746), attempt #2...
[2024-09-05 00:23:24.226248] > Doing something (time = 2024-09-05 00:22:29.753), attempt #3...
[2024-09-05 00:23:24.226261] > Doing something (time = 2024-09-05 00:22:34.760), attempt #4...
[2024-09-05 00:23:24.226275] > Doing something (time = 2024-09-05 00:22:39.769), attempt #5...
I'm less concerned about the timestamps lining up (cronjob vs. python), but maybe that's a clue?!
Note: If I run >ts '[%Y-%m-%d %H:%M:%.S]' from the command line, the timestamp works as expected.
It seems that the problem is none of cron's business.
I've tried with the following code
import time
from datetime import datetime
for i in range(0,5):
print(datetime.now())
time.sleep(1)
When I execute it with python test.py everything works fine. But when I try to redirect to tc with pipe. Your problem occurs.
The problem is due to buffer, it seems that buffer in pipe is block buffer.
So, for your question, the solution is flushing the stdout
import time
from datetime import datetime
for i in range(0,5):
print(datetime.now(), flush=True)
time.sleep(1)
or
import time
import sys
from datetime import datetime
for i in range(0,5):
print(datetime.now())
sys.stdout.flush()
time.sleep(1)
here are some pipe buffer related answers:
How to turn off stdout buffering in a pipe?