ffmpeg-python tracking transcoding process

My question is based on https://github.com/kkroening/ffmpeg-python/blob/master/examples/show_progress.py Ideally, all I want is to keep track e.g 0 - 100 for the transcoding process, emitting a streaming yield response to my gRPC client. Technically, I do not need a progress bar. How can I provide my own socket to the ffmpeg subprocess and handle write events to it?

Solution

You can redirect "progress" to stdout using -progress pipe:1.
See How to redirect -progress option output of ffmpeg to stderr?

The hard part is to actually get the progress in percentage.

Start by counting the total number of frames using FFprobe as described here.
Execute FFmpeg as sub-process, redirect -progress to stdout.
Start a thread that reads text lines from stdout.
The thread looks for frame=xx, get the frame, and put it in a list (list of 1 element).
Execute a "main loop" for demonstrating the progress readings.
The loop sleeps 1 second, reads the last element from the queue, and prints the progress.

The code starts by building a synthetic video file input.mp4 - used as input.

Here is a "self contained" code sample:

import subprocess as sp
import shlex
import json
from threading import Thread
import time


def progress_reader(procs, q):
    while True:
        if procs.poll() is not None:
            break  # Break if FFmpeg sun-process is closed

        progress_text = procs.stdout.readline()  # Read line from the pipe

        # Break the loop if progress_text is None (when pipe is closed).
        if progress_text is None:
            break

        progress_text = progress_text.decode("utf-8")  # Convert bytes array to strings

        # Look for "frame=xx"
        if progress_text.startswith("frame="):
            frame = int(progress_text.partition('=')[-1])  # Get the frame number
            q[0] = frame  # Store the last sample


# Build synthetic video for testing:
################################################################################
sp.run(shlex.split('ffmpeg -y -f lavfi -i testsrc=size=320x240:rate=30 -f lavfi -i sine=frequency=400 -f lavfi -i sine=frequency=1000 -filter_complex amerge -vcodec libx265 -crf 17 -pix_fmt yuv420p -acodec aac -ar 22050 -t 30 input.mp4'))
################################################################################


# Use FFprobe for counting the total number of frames
################################################################################
# Execute ffprobe (to show streams), and get the output in JSON format
# Actually counts packets instead of frames but it is much faster
# https://stackoverflow.com/questions/2017843/fetch-frame-count-with-ffmpeg/28376817#28376817
data = sp.run(shlex.split('ffprobe -v error -select_streams v:0 -count_packets -show_entries stream=nb_read_packets -of csv=p=0 -of json input.mp4'), stdout=sp.PIPE).stdout
dict = json.loads(data)  # Convert data from JSON string to dictionary
tot_n_frames = float(dict['streams'][0]['nb_read_packets'])  # Get the total number of frames.
################################################################################

# Execute FFmpeg as sub-process with stdout as a pipe
# Redirect progress to stdout using -progress pipe:1 arguments
process = sp.Popen(shlex.split('ffmpeg -y -loglevel error -i input.mp4 -acodec libvorbis -vcodec libvpx-vp9 -crf 20 -pix_fmt yuv420p -progress pipe:1 output.webm'), stdout=sp.PIPE)

q = [0]  # We don't really need to use a Queue - use a list of of size 1
progress_reader_thread = Thread(target=progress_reader, args=(process, q))  # Initialize progress reader thread
progress_reader_thread.start()  # Start the thread

while True:
    if process.poll() is not None:
        break  # Break if FFmpeg sun-process is closed

    time.sleep(1)  # Sleep 1 second (do some work...)

    n_frame = q[0]  # Read last element from progress_reader - current encoded frame
    progress_percent = (n_frame/tot_n_frames)*100   # Convert to percentage.
    print(f'Progress [%]: {progress_percent:.2f}')  # Print the progress


process.stdout.close()          # Close stdin pipe.
progress_reader_thread.join()   # Join thread
process.wait()                  # Wait for FFmpeg sub-process to finish

Note:

The code sample assumes that ffmpeg and ffprobe are in the executable path.

Sample output:

Progress [%]: 7.33
Progress [%]: 16.00
Progress [%]: 24.67
Progress [%]: 33.33
Progress [%]: 42.13
Progress [%]: 50.40
Progress [%]: 58.80
Progress [%]: 67.20
Progress [%]: 75.60
Progress [%]: 84.00
Progress [%]: 92.40
Progress [%]: 100.00