Search code examples
cpubenchmarkingramsnakemake

Monitoring RAM and CPU consumtion of Snakemake

I want to get CPU and RAM usage of a Snakemake pipeline across time. I run my pipeline on a slurm managed cluster. I know that Snakemake include benchmarking functions but they only reports pic consumption. Ideally, I would like to have an output file looking like this :

t  CPU     RAM
1  103.00  32
2  ...     ...

Is there any program to do so? Thanks!

Solution

  • Don't know any program already doing this, but you can monitor the CPU and MEM usage via native unix commmands, this post gives an answer that could fit your requirements.
    Here is a summary of the answer modified for this context:

    You can use this bash function

    logsnakemake() { while sleep 1; do  ps -p $1 -o pcpu= -o pmem= ; done; }

    You can tweak the frequency of logging by modifying the value of sleep.
    To log your snakemake process with pid=123 just type in the terminal:

    $ logsnakemake 123 | tee /tmp/pid.log