fluent-ffmpeg vstack passthrough stream of buffered images

Question

I am all day stuck at this issue. I have an array of buffered images and i need a script to concat them on top of each other.

I tried to do this using the sharp package, but since I am already using fluent-ffmpeg in the project for other reasons, I would like to stay with it. Also, i could not install the package correctly on m1 mac.

The current error I am getting is this Invalid file index 1 in filtergraph description [0:v]scale=iw:ih[v0];[1:v]scale=iw:ih[v1];[2:v]scale=iw:ih[v2];[3:v]scale=iw:ih[v3];vstack=4. And I must say, it completely makes sense, because there is really just one input object - the PassThrough stream. But the stream contains the array of the inputs, am I right?

return new Promise((resolve, reject) => {
        const posterOutputPath = `${outputPath}/output.jpg`;

        const inputStream = new PassThrough();
        buffers.forEach(buffer => { inputStream.write(buffer); });
        inputStream.end();

        let filter = '';
        buffers.forEach((buffer, index) => {
            filter += `[${index}:v]scale=iw:ih[v${index}];`;
        });
        filter += `vstack=${buffers.length}`;

        console.log(filter);

        ffmpeg(inputStream)
            .inputFormat('image2pipe')
            .complexFilter(filter)
            .outputOptions(['-frames:v 1', '-pix_fmt yuv420p'])
            .output(posterOutputPath)
            .on('end', () => {
                console.log('Images concatenated successfully');
                resolve(posterOutputPath);
            })
            .on('error', (err, stdout, stderr) => {
                console.error('Error concatenating images:', err);
                console.log("ffmpeg stdout:", stdout);
                console.log("ffmpeg stderr:", stderr);
                reject(err);
            })
            .run();

I've already tried creating images from the buffered input images to make sure that they are not empty.

Answer 1

For this specific use case, you can use the tile filter.

ffmpeg -f image2pipe -i pipe:0 -vf tile=layout=1x4 -frames:v 1 -pix_fmt yuv420p output.jpg

It can work here because all stacked images are expected to have the same dimensions and pixel format. Just don't put a very high number for rows or columns as that may breach the image size limits.