Processing Camera stream in Opencv, pushing it over RTMP (NGINX RTMP Module) using FFMPEG

Question

Output video: https://youtu.be/VxfoBQjoY6E

Explanation:

I want to: Process camera stream in Opencv and push it over to RTMP server. I already have NGINX (RTMP module) set up and I have tested streaming videos with both RTMP (Flash Player) and HLS.

I am reading the frames in a loop and using 'subprocess' in python to execute ffmpeg command. Here's the command I am using:

command = [ffmpeg,
    '-y',
    '-f', 'rawvideo',
    '-vcodec','rawvideo',
    '-pix_fmt', 'bgr24',
    '-s', dimension,
    '-i', '-',
    '-c:v', 'libx264',
    '-pix_fmt', 'yuv420p',
    '-preset', 'ultrafast',
    '-f', 'flv',
    'rtmp://10.10.10.80/live/mystream']


import subprocess as sp
...
proc = sp.Popen(command, stdin=sp.PIPE,shell=False)
...
proc.stdin.write(frame.tostring()) #frame is read using opencv

Problem:

I can see the stream fine but it freezes and resumes frequently. Here's the output of FFMPEG terminal log:

Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (libx264))
frame=  117 fps= 16 q=22.0 size=     344kB time=00:00:04.04 bitrate= 697.8kbits/s speed=0.543x   

It mentions speed at the end. I believe it should be close to 1x. I am not sure how to achieve that.

And I am on the same network as server, I can post my python code if required. Need some ffmpeg guru to give me some advise.

EDIT

My input fps is actually ~3. With '-use_wallclock_as_timestamps', '1' I can see in the log that speed is close to 1x. But HLS is not streaming live there's ~2 min delay, it halts and . Chris's advise partially worked. I am not sure where exactly is the problem, I am starting to believe it has something to do with nginx-rtmp module.

Here's the final output, on left it's flash and on right it's hls. I am showing the ffmpeg options at the end. https://youtu.be/jsm6XNFOUE4

Answer 1

I've had similar issues before when streaming either raw video (like you are) or from an MJPEG source. There are two input options to try with ffmpeg to get it to stay at 1x speed:

Option 1

ffmpeg -re -i <rest of input options>

-re tells ffmpeg to read at the native input rate

Option 2

ffmpeg -use_wallclock_as_timestamps 1 -i <rest of input options>

-use_wallclock_as_timestamps tells ffmepg to just take each frame as it comes in, take the system time, and make that the timestamp. I've found this option most effective when the speed slows right down.

Do regardless

Ensure you're encoding at a constant framerate. ffmpeg can get a bit finicky when the framerate varies, so on the output options, use -r 25 (replacing the 25 for your desired output framerate) to force ffmpeg to use a static framerate out