I have a pipeline to stream images over UDP
fps = 20
width = 500
height = 500
out_send = cv2.VideoWriter(
"appsrc ! videoconvert ! "
"video/x-raw,format=I420 ! "
"jpegenc ! rtpjpegpay !"
"udpsink host=127.0.0.1 port=5000",
cv2.CAP_GSTREAMER, 0, fps, (width, height), True
)
while True:
frame = np.random.randint(255, size=(height, width, 3), dtype=np.uint8)
out_send.write(frame)
time.sleep(0.05)
This starts the pipeline but I am unable to receive using the following pipeline. The streaming wont begin, just hangs.
gst-launch-1.0 udpsrc port=5000 ! application/x-rtp,media=video,payload=26,clock-rate=90000,encoding-name=JPEG ! rtpjpegdepay ! jpegdec ! videoconvert ! queue ! xvimagesink
However if the frame is captured from webcam as below
cap_send = cv2.VideoCapture(0)
fps = int(cap_send.get(cv2.CAP_PROP_FPS))
width = int(cap_send.get(cv2.CAP_PROP_FRAME_WIDTH))
height = int(cap_send.get(cv2.CAP_PROP_FRAME_HEIGHT))
while True:
ret, frame = cap_send.read()
if not ret:
break
out_send.write(frame)
Then the receiving pipeline starts and streams without any issues.
In both cases I see the following message on the receiving end
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Edit:
I ran the receiving pipeline with GST_DEBUG=3
and see the following output
gst_rtp_jpeg_depay_process:<rtpjpegdepay0> discarding data packets received when we have no header
This only happens if the sending pipeline uses raw images / numpy arrays as above.
What am I doing incorrect here. Very new to gstreamer and need some help
Goal: To stream images generated in opencv, asynchronously.
@Micka pointed me in the right direction.
The issue was with frame sizes. I have tried MJPEG and H264 and both need frame sizes (width/height) that are multiples of 8. Gstreamer will round them up to the closest 8th multiple. If the height is rounded then there is jitter seen at the end of the frame. If the width is rounded then a green screen is seen at the receiver.