Search code examples
openglgraphicsffmpegsdlsdl-2

Portable YUV Drawing Context

I have a stream of YUV data (from a video file) that I want to stream to a screen in real time. (Basically, I want to write a program that plays the video in real time.)

As such, I am looking for a portable way to send YUV data to the screen. I would ideally like to use something portable so I don't have to reimplement it for every major platform.

I have found a few options, but all of them seem to have significant issues. They are:

  1. Use OpenGL directly, converting the YUV data to RGB. (And using the single quad for the whole screen trick.)

This obviously won't work because converting RGB to YUV on the CPU is going to be too slow for displaying images in real time.

  1. Use OpenGL, but use a shader to convert the YUV stream to RGB.

This option is a bit better. Although the problem here is that (afaict), this will involve making two streams and splicing them together. It might work, but may have issues with larger resolutions.

  1. Instead use SDL, which has the option of creating a YUV context directly.

The problem with this is I already am using a cross platform widget library for other aspects of my program (such as playback controls). As far as I can tell, SDL only opens up in its on (possibly borderless) window. I would ideally like my controls and drawing context to be in the same window. Which I can do with opengl, but not SDL.

  1. Use SDL, and also use something like Qt for the on screen widgets, use a message passing protocol to communicate between the two libraries. Have the (borderless) SDL window constantly move itself on top of the opengl window.

While this approach is clever, it seems like the two windows could get out of sink easily making the user experience sub-optimal.

  1. Forget a cross platform library, do thinks OS specific, making use of hardware acceleration if present.

This is a fine solution although its not cross platform.

As such, is there any good way to draw YUV data to a screen that ideally is:

  1. Portable (at least to the major platforms).
  2. Fast enough to be real time.
  3. Allows other widgets in the same window.
Solution

  • Use the option number 2. There's no problem in doing the YUV to RGB conversion in the shader. There's no such other "portable" way to do that.

    Think like this: no matter "how big or small" your video is, the fragment shaders (where the conversion is done) will execute per pixel at the moment of the display, so you can have either a small video in full screen or big one, the computation (for the shaders) is the same, because they are displaying the same number of pixels.

    Any video card in normal conditions will be able to run this kind of shader without any problem.