So, I have a python script which outputs some data into terminal from time to time. Im trying to run in on the Ubuntu VPS even after I close the SSH connection and still keep the logs somewhere.
Im saving the logs by using:
python3 my_script.py >>file.txt
and it works perfect, however when I try to run this process using
nohup python3 my_script.py >>file.txt &
so it runs in the background and after the ssh connection is closed it seems to save only the first log outputted from my_script.py. I've also tried running this in crontab but the result is similar - only the first log is saved.
Any tips? What am I doing wrong?
I could not understand what you mean "the first log". Maybe the first line of logs?
To run something in the background when SSH connection is closed, I prefer Linux screen, a terminal simulation tool that help you run your command in a sub-process. With it, you could choose to view your output any time in the foreground, or leave your process run in the background.
screen is not included in most Linux distributions. Install it (Ubuntu):
$ sudo apt-get install screen
Run your script in the foreground:
$ screen python3 my_script.py
You'll see it running. Now detach from this screen: Press keys Ctrl-A followed by Ctrl-D. You'll be back to your shell where you run previous screen command. If you need to switch back to the running context, use screen -r command.
This tool supports multiple parallel running process too.
I've tried to redirect stdout or stderr to a file with > or >> symbol. It turned out in failure. I am not an expert of this either, and maybe you need to see its manual page. However, I tend to directly write to a file in Python scripts, with some essential output lines on the console.