Bash loop over files in directory outputting non existent files

Question

I am making mp4 files from a series of images. The images should run about 1 hour but I cannot get a full video because FFMPEG looks for a file it should not have.

[image2 @ 0x55fe14bef700] Could not open file : /mnt/run/image-005437.jpeg

What is confusing is that the list that I pass to ffmpeg should not include that file. If my script does not like the result it sends the file to a sub directory in the target folder called failed

the location of the file with that number

$ pwd
run/failed
]$ ls
failed-005437.jpeg 

the command I use to launch ffmpeg is the following

image_folder=$1
singularity exec --bind $work_dir:/mnt $work_dir/longleaf2.sif ffmpeg -f concat -safe 0 -y -i <(for f in /mnt/processed_images/$image_folder/image-%06d.jpeg; do echo "file '$f'";done) -vf "crop=trunc(iw/3)*2:trunc(ih/2)*2" -movflags +faststart /mnt/run/summary_files/${image_folder}.mp4

I have checked processed images and it is not there so why is ffmpeg looking for it?

pastebin of the failed run

https://pastebin.com/pF5ZefLf

My check that the file is not in the folder reference by the for loop so that it should never cause an error

$ ls image-005437.*
ls: cannot access image-005437.*: No such file or directory

Answer 1

Problem

When you run:

for f in /mnt/processed_images/$image_folder/image-%06d.jpeg; do echo "file '$f'";done

It will output:

file '/mnt/processed_images/foo/image-%06d.jpeg'

So then ffmpeg will use the sequence pattern type for the image demuxer. This expects a contiguous sequence.

Solution 1: Glob

Use a glob:

for f in /mnt/processed_images/$image_folder/*.jpeg; do echo "file '$f'";done

Now it will output each file. In this example image-000003.jpeg does not exist so it does not get listed:

file '/mnt/processed_images/foo/image-000001.jpeg'
file '/mnt/processed_images/foo/image-000002.jpeg'
file '/mnt/processed_images/foo/image-000004.jpeg'

Solution 2: Simplify and skip concat

Even better is to simplify your command by using the glob pattern type for the image demuxer within ffmpeg itself, and then you can avoid the concat demuxer:

image_folder=$1
singularity exec --bind "$work_dir":/mnt "$work_dir"/longleaf2.sif ffmpeg -pattern_type glob -framerate 25 -i "/mnt/processed_images/$image_folder/*.jpeg" -vf "crop=trunc(iw/3)*2:trunc(ih/2)*2,format=yuv420p" -movflags +faststart /mnt/run/summary_files/${image_folder}.mp4

• The image demuxer glob pattern is not available for Windows users.
• Added the -framerate input option for the image demuxer.
• Added the format filter to make YUV 4:2:0 chroma subsampling for compatibility.
• The variables have been quoted. See shellcheck.net.
• FFmpeg 4.1 release branch is old. Download or compile a modern version before doing anything else.