Search code examples
androidopengl-esandroid-mediacodechdr

Process HDR10 video on Android with MediaCodec


I want to do tone-mapping of HDR10 on Android use OpenGL ES, the first thing is to get the frame data. On Android after video decoded by mediacodec, its data is on an external oes texture, I want to know what the internal format of the texture is when decoding hdr10 video.

Can I get 10bit data from this external oes texture, and if possible, how to handle the 10bit data in OpenGL ES (using float texture?).

I have tried to query the format of the texture, but failed, the value reported seems not correct. And so as the width and height of the texture.

glGetTexLevelParameteriv(GL_TEXTURE_EXTERNAL_OES, 0, GL_TEXTURE_INTERNAL_FORMAT, &format);

Solution

  • In general for YUV you need to use an external sampler, which will include color conversion to RGB. There is no standard for what the Android memory layout is for YUV surfaces, in particular for 10-bit where there are multiple competing "standards", so anything you do here which touches the raw YUV data is going to be non-portable and device specific.

    Does the rgb value I get match the 10 bit value in HDR video?

    It should be a color converted representation of the 10-bit value, assuming it's been converted correctly. I obviously have no way to verify it, and there isn't a tight specification here. Could you just be getting an 8-bit equivalent? Yes, that's certainly possible.