run multiple java apps at the background and check the exit code if any

I am implementing a JAR runner script in a Docker container. I pick up all the JAR files in a specific directory (in a Docker volume) and run all of them after each other as a background process. If the JAR file does not start properly, then I need to show an error on the console and log the exit code.

This is the code of the runner bash script (run-jars.sh):

number_of_jars=$(find "$JAR_HOME" -maxdepth 1 -type f -name "*.jar" | wc -l)
printf "%s | [DEBUG] JAR files in the %s directory: %s\n" "$(date +"%Y-%b-%d %H:%M:%S")" "$JAR_HOME" "$number_of_jars"

for jar_file in "$JAR_HOME"/*.jar; do
   printf "%s | [INFO]  starting %s at the background...\n" "$(date +"%Y-%b-%d %H:%M:%S")" "$jar_file"
   nohup java -jar "$jar_file" 2>&1 &

   exit_code=$?
   pid=$!
   printf "%s | [DEBUG] PID       of the %s is %s\n" "$(date +"%Y-%b-%d %H:%M:%S")" "$jar_file" "$pid"
   printf "%s | [DEBUG] Exit code of the %s is %s\n" "$(date +"%Y-%b-%d %H:%M:%S")" "$jar_file" "$exit_code"

   timeout 5 tail --pid=$pid -f /dev/null
   exit_code=$?
   printf "%s | [DEBUG] Exit code of the %s is %s\n" "$(date +"%Y-%b-%d %H:%M:%S")" "$jar_file" "$exit_code"
   if [ "$exit_code" -ne 0 ]; then
      printf "%s | [ERROR] Java application %s exited with an error: %s" "$(date +"%Y-%b-%d %H:%M:%S")" "$jar_file" "$exit_code"
   fi
done

The loop works fine and JARs has been started but my exit-code checker does not work. The exit code I get is allways ZERO despite one JAR is corrupt deliberately.

Log:

2024-Mar-03 13:38:24 | [DEBUG] JAR files in the /jars-to-run directory: 2
2024-Mar-03 13:38:24 | [INFO]  running /jars-to-run/aa.jar at the background...
Error: Invalid or corrupt jarfile /jars-to-run/aa.jar
2024-Mar-03 13:38:24 | [DEBUG] PID       of the /jars-to-run/aa.jar is 139
2024-Mar-03 13:38:24 | [DEBUG] Exit code of the /jars-to-run/aa.jar is 0
2024-Mar-03 13:38:24 | [DEBUG] Exit code of the /jars-to-run/aa.jar is 0
2024-Mar-03 13:38:24 | [INFO]  running /jars-to-run/rest-template-0.0.2.jar at the background...
2024-Mar-03 13:38:24 | [DEBUG] PID       of the /jars-to-run/rest-template-0.0.2.jar is 146
2024-Mar-03 13:38:24 | [DEBUG] Exit code of the /jars-to-run/rest-template-0.0.2.jar is 0

Solution

You cannot collect the exit status of a background process immediately after launching it.

Try this :

declare -A pids
for jar_file in "$JAR_HOME"/*.jar; do
    nohup java -jar "$jar_file" 2>&1 &
    pids[$!]="$jar_file"
done
while test -n "${pids[*]}" && { wait -n -p pid; status=$?; }; do
    echo "java process with jar:[${pids[$pid]}] and pid:$pid exited with status:$status"
    unset "pids[$pid]"
done