How to use hardware decoder for decode RTSP stream in python? (NVidia JetSon Nano)

Question

I have NVIDIA Jetson Nano and FullHD Ip camera. Camera streams RTSP/h264.

I want to decode frames in python script from this camera for analizies.

So, i tried use something like that:

# import the necessary packages
from imutils.video import VideoStream
import imutils
import time
import cv2
# grab a reference to the webcam
print("[INFO] starting video stream...")
#vs = VideoStream(src=0).start()
vs = VideoStream(src="rtsp://login:[email protected]").start()
time.sleep(2.0)

# loop over frames
while True:
    # grab the next frame
    frame = vs.read()
    # resize the frame to have a maximum width of 500 pixels
    frame = imutils.resize(frame, width=500)
    # show the output frame
    cv2.imshow("Frame", frame)
    key = cv2.waitKey(1) & 0xFF
    # if the `q` key was pressed, break from the loop
    if key == ord("q"):
        break
# release the video stream and close open windows
vs.stop()
cv2.destroyAllWindows()

That's works, but in that way frame decoded on CPU. How to use GPU decoder?

Answer 1

The solution:

Use cv2.VideoCapture with GStreamer backend:

import cv2
pipeline = "rtspsrc location=\"rtsp://login:password@host:port/\" ! rtph264depay ! h264parse ! omxh264dec ! nvvidconv ! video/x-raw, format=(string)BGRx! videoconvert ! appsink"
capture = cv2.VideoCaputure(pipeline, cv2.CAP_GSTREAMER)

while capture.isOpened():
    res, frame = capture.read()
    cv2.imshow("Video", frame)
    key = cv2.waitKey(1) & 0xFF
    if key == ord("q"):
        break
capture.release()
cv2.destroyAllWindows()