Pipe and OpenCV to FFmpeg with audio streaming RTMP in Python

Question

I'm trying to stream FFmpeg with audio.
I will show my code below:

Import module

import subprocess as sp

---

Create variables

rtmpUrl = "rtmp://a.rtmp.youtube.com/live2/key"
camera_path = "BigBuckBunny.mp4"
cap = cv.VideoCapture(camera_path)

# Get video information
fps = int(cap.get(cv.CAP_PROP_FPS))
width = int(cap.get(cv.CAP_PROP_FRAME_WIDTH))
height = int(cap.get(cv.CAP_PROP_FRAME_HEIGHT))

---

Command param

# ffmpeg command
command = ['ffmpeg',
        '-y',
        '-f', 'rawvideo',
        '-vcodec','rawvideo',
        '-pix_fmt', 'bgr24',
        '-s', "{}x{}".format(width, height),
        '-r', str(fps),
        '-i', '-',
        '-c:v', 'libx264',
        '-pix_fmt', 'yuv420p',
        '-preset', 'ultrafast',
        '-f', 'flv', 
        rtmpUrl]

---

Create subprocess to ffmpeg command

# Pipeline configuration
p = sp.Popen(command, stdin=sp.PIPE)

---

Send frame to RTMP server

# read webcamera
while(cap.isOpened()):
    ret, frame = cap.read()
    if not ret:
        print("Opening camera is failed")
        break

    # write to pipe
    p.stdin.write(frame.tobytes())

I hope you can help me to be able to live stream via FFmpeg over RTMP with audio. Thanks!

Answer 1

Assuming you actually need to use OpenCV for the video, you have to add the audio directly to FFmpeg as Gyan commented, because OpenCV does not support audio.

-re argument is probably required for live streaming.

---

For testing, I modified the RTMP URL from YouTube to localhost.
FFplay sub-process is used for capturing the stream (for testing).

Complete code sample:

import subprocess as sp
import cv2

#rtmpUrl = "rtmp://a.rtmp.youtube.com/live2/key"
rtmp_url = "rtmp://127.0.0.1:1935/live/test"  # Use localhost for testing
camera_path = "BigBuckBunny.mp4"
cap = cv2.VideoCapture(camera_path)

# Get video information
fps = int(cap.get(cv2.CAP_PROP_FPS))
width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))

# Start the TCP server first, before the sending client (for testing).
ffplay_process = sp.Popen(['ffplay', '-listen', '1', '-i', rtmp_url])  # Use FFplay sub-process for receiving the RTMP video.

# ffmpeg command
# OpenCV does not support audio.
command = ['ffmpeg',
        '-y',
        '-re', # '-re' is requiered when streaming in "real-time"
        '-f', 'rawvideo',
        #'-thread_queue_size', '1024',  # May help https://stackoverflow.com/questions/61723571/correct-usage-of-thread-queue-size-in-ffmpeg
        '-vcodec','rawvideo',
        '-pix_fmt', 'bgr24',
        '-s', "{}x{}".format(width, height),
        '-r', str(fps),
        '-i', '-',
        '-vn', '-i', camera_path,  # Get the audio stream without using OpenCV
        '-c:v', 'libx264',
        '-pix_fmt', 'yuv420p',
        '-preset', 'ultrafast',
        # '-c:a', 'aac',  # Select audio codec
        '-bufsize', '64M',  # Buffering is probably required
        '-f', 'flv', 
        rtmp_url]

# Pipeline configuration
p = sp.Popen(command, stdin=sp.PIPE)

# read webcamera
while (cap.isOpened()):
    ret, frame = cap.read()
    if not ret:
        print("End of input file")
        break

    # write to pipe
    p.stdin.write(frame.tobytes())

p.stdin.close()  # Close stdin pipe
p.wait()

ffplay_process.kill()  # Forcefully close FFplay sub-process