opengl es yuv to rgb conversion display only green and pink colors

Question

I am doing yuv to rgb conversion using opengl shaders. But its only show green and pink colors. I am using ffmpeg to decode movie. I am beginner in this, so have no idea how to solve it. ffmpeg give me three buffers of yuv. I am directly assigning these buffers to three textures.

Here are shaders I am using.

static const char* VERTEX_SHADER =
"attribute vec4 vPosition;                      \n"
"attribute vec2 a_texCoord;                     \n"
"varying vec2 tc;                               \n"
"uniform mat4 u_mvpMat;                         \n"
"void main()                                    \n"
"{                                              \n"
"   gl_Position = u_mvpMat * vPosition;         \n"
"   tc = a_texCoord;                            \n"
"}                                              \n";

static const char* YUV_FRAG_SHADER =
"#ifdef GL_ES                                   \n"
"precision highp float;                         \n"
"#endif                                         \n"
"varying vec2 tc;                               \n"
"uniform sampler2D TextureY;                    \n"
"uniform sampler2D TextureU;                    \n"
"uniform sampler2D TextureV;                    \n"
"uniform float imageWidth;                      \n"
"uniform float imageHeight;                     \n"

"void main(void)                                \n"
"{                                              \n"
    "float nx, ny;                              \n"
    "vec3 yuv;                                  \n"
    "vec3 rgb;                                  \n"
    "nx = tc.x;                                 \n"
    "ny = tc.y;                                 \n"

    "yuv.x = texture2D(TextureY, tc).r;         \n"
    "yuv.y = texture2D(TextureU, vec2(nx/2.0, ny/2.0)).r - 0.5;   \n"
    "yuv.z = texture2D(TextureV, vec2(nx/2.0, ny/2.0)).r - 0.5;   \n"

    // Using BT.709 which is the standard for HDTV
    "rgb = mat3( 1,              1,      1,     \n"
                "0,       -0.18732, 1.8556,     \n"
                "1.57481, -0.46813,      0)*yuv;\n"

    // BT.601, which is the standard for SDTV is provided as a reference
    //"rgb = mat3( 1,            1,     1,        \n"
    //            "0,     -0.34413, 1.772,        \n"
    //            "1.402, -0.71414,     0) * yuv; \n"
    "gl_FragColor = vec4(rgb, 1.0);             \n"
"}                                              \n";

Output:

What wrong am I doing? Please help me out with this.

Thank You.

UPDATE:

While debugging ffmpeg decoding, I found that ffmpeg decoder give PIX_FMT_YUV420P output format. Do I have to make some tweaks to get correct image colors?

Answer 1

I'm not sure about this transformation:

"rgb = mat3( 1,              1,      1,     \n"
            "0,       -0.18732, 1.8556,     \n"
            "1.57481, -0.46813,      0)*yuv;\n"

In refreshing my memory on matrix * vector operations in GLSL using this page as reference, I think you either need to transpose the coefficient matrix, or move yuv to the front of the operation, i.e., yuv * mat3(...). Performing the operation as mat3(...) * yuv means:

r = y * 1 + u * 1 + v * 1
g = y * 0 + u * -0.18732 + v * 1.8556
b = y * 1.57481 + u * -0.46813 + v * 0

And these conversions are very incorrect.

As another reference, here's a small, complete, sample GLSL shader program that converts YUV -> RGB that may be of some guidance: http://www.fourcc.org/source/YUV420P-OpenGL-GLSLang.c