Spinning up multiple tmux sessions in bash

Question

I have a python script, my_python_script that takes an input date. I have a bash script that calls this python script for a series of dates, within a for loop. I tried creating a named session within the for loop, but this nests tmux sessions:

call_runner_date_range() {
    start=$1
    end=$2

    start=$(date -d $start +%Y%m%d)
    end=$(date -d $end +%Y%m%d)

    while [[ $start -le $end ]]
    do
            tmux new -s $start
            start=$(date -d"$start + 1 day" +"%Y%m%d")
            formatted_date=$(date -d $start +%m-%d-%Y)
            python -m my_python_script --analysis-date=$formatted_date
    done
}

call_runner_date_range '2016-12-31' '2019-08-15'

I would like to modify this bash script to open a new tmux session for each date (the reason I want to use tmux is so I can monitor the output logs), so that these scripts can run in parallel. How can I do this, and is there a preferable alternative to using tmux?

Answer 1

To start a process in the background use the & as the terminating character of the command. Use > to redirect the standard output of a process to a file. For some reference, you could see bash manual List of commands or bash manual redirections but many resources are available online.

So the script would just become:

call_runner_date_range() {
    start=$1
    end=$2

    start=$(date -d $start +%Y%m%d)
    end=$(date -d $end +%Y%m%d)

    while [[ $start -le $end ]]
    do
            start=$(date -d"$start + 1 day" +"%Y%m%d")
            formatted_date=$(date -d $start +%m-%d-%Y)
            python -m my_python_script --analysis-date=$formatted_date > log_$start.log &
    done
}

call_runner_date_range '2016-12-31' '2019-08-15'

You can use wait shell builtin to wait for all currently still running background processes in current bash shell. Or, you can save each background job process id PID using $! and specify the list of pids you want to wait for.

childs=() # bash array, initializing to empty

# then later:
python -m my_python_script --analysis-date=$formatted_date > log_$start.log &
childs+=($!)  # append the background jobs pids name to bash array

# then later
wait "${childs[0]}" # wait for first background job in childs array
wait "${childs[@]}" # wait for all background jobs in childs array

Note you may interested in using xargs that can (on most implementations) execute commands in parallel for each line of input or the amazing GNU parallel which is just a shell tool to execute multiple jobs in parallel as easy as possible.