Search code examples
pythonvideocpugstreamer

Gstreamer either getting high CPU or returning older frames

I am trying to decode several video streams using GStreamer and Python. Here's my code:

self.video_capture = cv2.VideoCapture(self.pipeline, cv2.CAP_GSTREAMER)
if self.video_capture.isOpened() is False:
    raise Error
while True:
    status, image = self.video_capture.read()
    # do slow stuff

I am encountering the following issue:

Option 1

If I use the pipeline:

self.pipeline = f"rtspsrc location={self.url} latency=10 ! rtph264depay ! h264parse ! avdec_h264 ! videoconvert ! appsink max-buffers=1 drop=true"

I get extremely high CPU consumption. Also, I am probably decoding more frames than the number my application can actually consumes.

Option 2

If I use this pipeline

self.pipeline = f"rtspsrc location={self.url} latency=10 ! rtph264depay ! h264parse ! avdec_h264 ! videoconvert ! appsink max-buffers=1 drop=false"

After a few hours the application is returning me old frames. E.g. at 9am my application was receiving frames at nighttime (from the night before).

Is this the expected behavior? How can I solve this problem?

Thank you a lot

Solution
    • With drop=true we decode all the frames and we drop them only after decoding them
    • Probably best solution would be to use both latency=10 and drop=true when using software decoders (this could cause corrupted frames with hardware decoders)