Search code examples
pythonbashtmuxsnakemakenohup

How to run snakemake on linux backend without output in putty?


I try to run all the rules by followed commands:

touch scripts/*.py
snakemake --cores <YOUR NUMBER>

The problem is my local internet connection is unstable, I could submit the command through putty to the linux computation platform, while it seems that there're always outputs returns back to the putty interface. So when my local internet connection is interrupted, the code running is also interrupted.

Is there any methods that I could let the codes just run on the linux backends? Then outputs could be written in the log file at last.

This could be a very basic question.


Solution

  • This is a common problem (not just for snakemake), and there are several options, at least the following:

    • use a program that can persists across multiple connection: popular options are screen, tmux. The workflow would look like this: log on to the server, launch screen or tmux, once inside the program launch the code you would like to run, log off, next time you login to the server, you can reconnect to the previous session and observe computations that were done in the meantime. I recommend tmux, see this tmux tutorial.

    • use nohup, this launches the computation in the background and it will continue running on the server if you disconnect:

      nohup snakemake --cores <YOUR NUMBER>
      

      Note that with this option, if you want to see the progress of computation, you will need to watch the appropriate .log inside the .snakemake folder.