I'm writing a bash script to execute a python program with different values of parameters 'H' and 'K'. What I would like to happen is that only one instance of 'program.py' runs at a time and subsequent jobs are not started until the previous one finishes.
Here's what I tried -
#!/bin/bash
H="1 10 100"
K="0.01 0.1"
for H_val in ${H}
do
for K_val in ${K}
do
{ nohup python program.py $H_val $K_val; } &
done
done
However, this just loops over all the parameter values without waiting for any one job to finish. Conversely, if I modify the above slightly by taking off the ampersand, I can run each job individually - but not in the background. Any ideas on how to proceed?
Put the & on the command you want to put in the background. In this context, that might be your whole loop:
#!/usr/bin/env bash
# ^^^^- ensures that array support is available
hValues=( 1 10 100 ) # best practice is to iterate over array elements, not...
kValues=( 0.01 0.1 ) # ...words in strings!
{
# perform conditional redirections akin to what nohup would do
[ -t 0 ] && exec </dev/null # stdin
[ -t 1 ] && exec >myprogram.log # stdout
[ -t 2 ] && exec 2>&1 # stderr
for hVal in "${hValues[@]}"; do
for kVal in "${kValues[@]}"; do
python program.py "$hVal" "$kVal"
done
done
} &
Notice how the & was moved to be after the done -- that way we background the entire loop, not a single command within it, so the backgrounded process -- running that loop -- invokes only one copy of the Python interpreter at a time.
The redirections (</dev/null, >myprogram.log, and 2>&1) are equivalent to how nohup would redirect stdout and stderr to nohup.out if they weren't already going to a file (or other non-TTY destination) to ensure that they aren't attached to a TTY; adjust the name myprogram.log to your preference.