Bash While loop read file by line exit on certain line

Question

I'm writing a simple Bash script that simply the call of HadnBrakeCli for render videos. I also implemented a simple queue option: the queue file just store the line-command it has to call to start a render. So I wrote a while-loop to read one line at time, eval $line and repeat untill file ends.

    if [[ ${QUEUE_MODE} = 'RUN' ]]; then
    QUEUE_LEN=`cat ${CONFIG_DIR}/queue | wc -l`
    QUEUE_POS='1'
    printf "Queue lenght:\t ${QUEUE_LEN}\n"

    while IFS= read line; do
        echo "--Running render ${QUEUE_POS} on ${QUEUE_LEN}..."
        echo "++" && echo "$line" && echo "++"
        eval "${line}"
        tail -n +2 "${CONFIG_DIR}/queue" > "${CONFIG_DIR}/queue.tmp" && mv "${CONFIG_DIR}/queue.tmp" "${CONFIG_DIR}/queue"
        echo "--Render ended"
        QUEUE_POS=`expr $QUEUE_POS + 1`
    done < "${CONFIG_DIR}/queue"
    exit 0

The problem is that any command makes the loop to work fine (empty line, echo "test"...), but as soon a proper render is loaded, it is launched and finished correctly, but also the loops exists.

I am a newbie so I tried some minor changes to see what effect I got, but nothing change the result. I commented the command tail -n +2 "${CONFIG_DIR}/queue" > "${CONFIG_DIR}/queue.tmp" && mv "${CONFIG_DIR}/queue.tmp" "${CONFIG_DIR}/queue" or I added/removed IFS= in the while-loop or removed the -r in read command. Sorry if the question is trivial, but I'm really missing some major part in how it works, so I have no idea even how to search for the solution.

I'll put a sample of a general render in the queue file. HandBrakeCLI -i "/home/andrea/Videos/done/Rap dottor male e mini me.mp4" -o "/hdd/Render/Output/Rap dottor male e mini me.mkv" -e x265 -q 23 --encoder-preset faster --all-audio -E av_aac -6 dpl2 --all-subtitles -x pmode:pools='16' --verbose=0 2>/dev/null

Answer 1

HandBrakeCLI reads from standard input, which steals the rest of the queue file before read line can see it. My favorite solution to this is to pass the file over something other than standard input, like file descriptor #3:

...
while IFS= read line <&3; do    # The <&3 makes it read from FD #3
    ...
done 3< "${CONFIG_DIR}/queue"   # The 3< redirects the file into FD #3

Another way to avoid the problem is to redirect input to the HandBrakeCLI command:

...
eval "${line}" </dev/null
...

There's some more info about this in BashFAQ #89: I'm reading a file line by line and running ssh or ffmpeg, only the first line gets processed!

Also, I'm not sure I trust the way you're using tail to remove lines from the queue file as they're executed. I'm not sure it's really wrong, it just looks fragile to me. Also, I'd recommend using lower- or mixed-case variable names, since there are a bunch of all-caps names with special meanings, and re-using one of them by mistake can have weird consequences. Finally, I'd recommend running your script through shellcheck.net, as it'll make some other good recommendations.

[BTW, this question is a duplicate of "Bash script do loop exiting early", but that doesn't have any upvoted or accepted answers.]