Search code examples
bashbackgroundjobs

How to get the correct number of background jobs running, f.ex. $(jobs | wc -l | xargs) returns 1 instead of 0


I have a jenkins build job that starts processes in the background. I need to write a function that checks wether there are still background processes running. To test it I came up with this:

#!/bin/bash -e

function waitForUploadFinish() {
  runningJobs=$(jobs | wc -l | xargs)
  echo "Waiting for ${runningJobs} background upload jobs to finish"

  while [ "$(jobs | wc -l | xargs)" -ne 0 ];do
    echo "$(jobs | wc -l | xargs)"
    echo -n "."     # no trailing newline
    sleep 1
  done
  echo ""
}


for i in {1..3}
do
  sleep $i &
done

waitForUploadFinish

The problem is it never comes down to 0. Even when the last sleep is done, there is still one job running?

mles-MBP:ionic mles$ ./jobs.sh 
Waiting for 3 background upload jobs to finish
3
.2
.1
.1
.1
.1
.1
.1

Why I don't want to use wait here

In the Jenkins build job script where this snippet is for, i'm starting background upload processes for huge files. They don't run for 3 seconds like in the example here with sleep. They can take up to 30 minutes to proceed. If I use wait here, the user would see something like this in the log:

upload huge_file1.ipa & 
upload huge_file2.ipa & 
upload huge_file3.ipa & 
wait

They would wonder why is nothing going on?

Instead I want to implement something like this:

upload huge_file1.ipa & 
upload huge_file2.ipa & 
upload huge_file3.ipa & 
Waiting for 3 background upload jobs to finish
............
Waiting for 2 background upload jobs to finish
.................
Waiting for 1 background upload jobs to finish
.........
Upload done

That's why I need the loop with the current running background jobs.


Solution

  • This fixes it:

    function waitForUploadFinish() {
      runningJobs=$(jobs | wc -l | xargs)
      echo "Waiting for ${runningJobs} background upload jobs to finish"
    
      while [ `jobs -r | wc -l | tr -d " "` != 0 ]; do 
        jobs -r | wc -l | tr -d " "
        echo -n "."     # no trailing newline
        sleep 1
      done
      echo ""
    }
    

    Note: you will only count the background processes that are started by this bash script, you will not see the background processes from the starting shell.

    As the gniourf_gniourf commented: if you only need to wait and don't need to output then a simple wait after the sleeps is much simpler.

    for i in {1..3}; do
      sleep $i &
    done
    
    wait