Multiple applications to access the same DeckLink device (openCV and rtmp pushing)? What options do I have?

Description:

I have a requirement that I must do two things to a single camera stream from a DeckLink device - OpenCV processing and RTMP live streaming. If it is relevant, the device is DeckLink 8K Pro.

The DeckLink device (one particular port) does not allow more than one application to access the video stream. My two applications are:

C++ OpenCV application which must run image processing algorithms on the video stream.
Unreal Media Server which must RTMP push the stream to a remote endpoint, which should redistribute the stream to clients.

Both applications must be running on the same Windows workstation.

Problem:

If either application is running, it has exclusive access to the video stream and the other application cannot read the stream, and there seems to be no solution to this problem.

So, I am really looking for a smart solution some of you may have implemented before or have an idea which might work for me.

My solution so far:

In this solution, Unreal Media Server is replaced with FFMPEG.

Since OP's effort must be put into the solution before posting here, I have been browsing through possible solutions the past week. I came up with an FFMPEG-based solution which is the only application which reads the DeckLink video stream and does exactly two things in a single command - pushes video packets via RTMP and creates a local UDP stream. And, OpenCV's VideoCapture class is able to pick up UDP stream. For testing, I have used Twitch as RTMP server and VLC as UDP tester. It works and it seems like a good solution except for the delay introduced in UDP streaming which is about 0.4 seconds. Unfortunately, I am not able to evaluate RTMP delay because Twitch introduces its own >5s delay on its own. But that is not a problem for now.

Here's the FFMPEG solution (Note I am using computer webcam here but same goes for DeckLink).

Steram via:

ffmpeg -threads:v 2 -threads:a 8 -filter_threads 2 -thread_queue_size 512 -y -f dshow -video_size 640x480 -pixel_format yuyv422 -framerate 30 -rtbufsize 100M -i video="HD WebCam" -f dshow -rtbufsize 100M -i audio="Microphone Array (Realtek High Definition Audio(SST))" -preset ultrafast -vcodec libx264 -tune zerolatency -b 900k -map 0:v:0 -f mpegts udp://127.0.0.1:5555 -pix_fmt yuv420p -c:v libx264 -qp:v 19 -profile:v high -rc:v cbr_ld_hq -level:v 4.2 -r:v 60 -g:v 120 -bf:v 3 -refs:v 16 -f flv rtmp://live-fra05.twitch.tv/app/stream_key

Play via:

ffplay -probesize 32 -sync ext udp://127.0.0.1:5555

I want to hear how you would improve my own solution to better fit the problem (e.g. less latency) or if you have a better solution than mine.

Thanks in advance.

Solution

Once the stream arrives to Unreal Media Server, you could start MPEG2-TS broadcasting that stream, using Unreal Media Server Configurator. Then your OpenCV's VideoCapture class will be able to pick up that MPEG2-TS UDP stream as well.