How to display a raw YUV frame in a Cocoa OpenGL program

Question

I have been assigned wit the task to write a program that takes a sample raw YUV file and display it in a Cocoa OpenGL program.

I am an intern at my job and I have little or no clue how to start. I have been reading wikipedia & articles on YUV, but I couldn't find any good source code on how to open a raw YUV file, extract the data and convert it into RGB and display it in the view window.

Essentially, I need help with the following aspects of the task -how to extract the YUV data from the sample YUV file -how to convert the YUV data into RGB color space -how to display the RGB color space in OpenGL. (This one I think I can figure out with time, but I really need help with the first two points)

please either tell me the classes to use, or point me to places where i can learn about YUV graphic/video display

Answer 1

This answer is not correct, see the other answers and comments. Original answer left below for posterity.

---

You can't display it directly. You'll need to convert it to an RGB texture. As you may have gathered from Wikipedia, there are a bunch of variations on the YUV color space. Make sure you're using the right one.

For each pixel, the conversion from YUV to RGB is a straightforward linear transformation. You just do the same thing to each pixel independently.

Once you've converted the image to RGB, you can display it by creating a texture. You need to call glGenTextures() to allocate a texture handle, glBindTexture() to bind the texture to the render context, and glTexImage2D() to upload the texture data to the GPU. To render it, you again call glBindTexture(), followed by the rendering of a quad with texture coordinates set up properly.

// parameters: image:  pointer to raw YUV input data
//             width:  image width (must be a power of 2)
//             height: image height (must be a power of 2)
// returns: a handle to the resulting RGB texture
GLuint makeTextureFromYUV(const float *image, int width, int height)
{
    float *rgbImage = (float *)malloc(width * height * 3 * sizeof(float));  // check for NULL
    float *rgbImagePtr = rgbImage;

    // convert from YUV to RGB (floats used here for simplicity; it's a little
    // trickier with 8-bit ints)
    int y, x;
    for(y = 0; y < height; y++)
    {
        for(x = 0; x < width; x++)
        {
            float Y = *image++;
            float U = *image++;
            float V = *image++;
            *rgbImagePtr++ = Y                + 1.13983f * V;  // R
            *rgbImagePtr++ = Y - 0.39465f * U - 0.58060f * V;  // G
            *rgbImagePtr++ = Y + 2.03211f * U;                 // B
        }
    }

    // create texture
    GLuint texture;
    glGenTextures(1, &texture);

    // bind texture to render context
    glBindTexture(GL_TEXTURE_2D, texture);

    // upload texture data
    glTexImage2D(GL_TEXTURE_2D, 0, 3, width, height, 0, GL_RGB, GL_FLOAT, rgbImage);

    // don't use mipmapping (since we're not creating any mipmaps); the default
    // minification filter uses mipmapping.  Use linear filtering for minification
    // and magnification.
    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);

    // free data (it's now been copied onto the GPU) and return texture handle
    free(rgbImage);
    return texture;
}

To render:

glBindTexture(GL_TEXTURE_2D, texture);

glBegin(GL_QUADS);
    glTexCoord2f(0.0f, 0.0f); glVertex3f( 0.0f,  0.0f, 0.0f);
    glTexCoord2f(1.0f, 0.0f); glVertex3f(64.0f,  0.0f, 0.0f);
    glTexCoord2f(1.0f, 1.0f); glVertex3f(64.0f, 64.0f, 0.0f);
    glTexCoord2f(0.0f, 1.0f); glVertex3f( 0.0f, 64.0f, 0.0f);
glEnd();

And don't forget to call glEnable(GL_TEXTURE_2D) at some point during initialization, and call glDeleteTextures(1, &texture) during shutdown.