Search code examples
linuxbashprocessparallel-processinggnu-parallel

How do I terminate GNU parallel without killing running jobs?


I'm running a bunch of shell scripts like parallel -a my_scripts bash and at some point I decided I've run enough of them and would like to stop spawning new jobs, and simply let all the existing jobs finish. Put another way, I want to kill the parent process without killing the children.

There seems to be ways of controlling termination when first launching GNU parallel (for example if I know in advance I only want to run x jobs, then I can use --halt now,success=x argument), but I couldn't find how to control GNU parallel when it is already running.

Sure I can just CTRL+C to kill parallel, and rerun the jobs that were aborted, but I thought there might be a smarter way.


Solution

  • Update:

    If you have a new version of parallel >= 20190322: as per @Bowi's comment, "since 2019, it is no longer SIGTERM, it's SIGHUP instead:

    kill -HUP $PARALLEL_PID
    

    SIGTERM now terminates the children without letting them finish."

    See:

    Original answer:

    I figured it out. The answer is to simply send SIGTERM to the parent parallel process (just killing its PID will do). parallel then responds with the following (in this case I have 4 jobs running):

    parallel: SIGTERM received. No new jobs will be started.
    parallel: Waiting for these 4 jobs to finish. Send SIGTERM again to stop now.
    

    I dug it out of the man page:

    COMPLETE RUNNING JOBS BUT DO NOT START NEW JOBS
    If you regret starting a lot of jobs you can simply break GNU parallel, but if you want to make sure you do not have half-completed jobs you should send the signal SIGTERM to GNU parallel:

    killall -TERM parallel

    This will tell GNU parallel to not start any new jobs, but wait until the currently running jobs are finished before exiting.