Search code examples
androidgstreameroculusquest

Enabling OMX support in GStreamer for Android

I'm currently trying to hook onto an HW H.264 decoder on Oculus Quest 2 (Adreno 650).

The platform supports just one HW decoder OMX.qcom.video.decoder.avc

The video decoding pipeline is created using GStreamer and currently looking like this: rtspsrc location=rtsp://192.168.1.239:8554/left ! application/x-rtp,encoding-name=H264,payload=96 ! rtph264depay ! decodebin ! videoconvert ! video/x-raw,width=1920,height=1080,format=RGBA ! appsink emit-signals=true name=wallesink sync=false

The decodebin bin should automatically use the available OMX decoder, but GStreamer library isn't built with it being supported.

I'm currently using this release of GStreamer.

So my question is simple: What is the easiest way to add support for said OMX decoder?

I've tried building from source with Meson, but was unable to do it with OMX enabled.

Are there any other builds for Android that have this decoder bundled?

Solution

  • Those codecs are prebuilt in the android binaries, distributed from gstreamer home page https://gstreamer.freedesktop.org/data/pkg/android/

    OMX decoder is available at least in gstreamer 1.18.6 and when using decodebin3 node it knew how to choose the HW decoder on quest1 and quest2.

    However with 1.20.4 automatic selecting didn't work out-of-box and one has to set higher ranking for omx decoders to make decodebin to select them.

    https://gstreamer.freedesktop.org/documentation/tutorials/playback/hardware-accelerated-video-decoding.html?gi-language=c

    Here I did find one C example how to do it from github: https://github.com/github0925/test/blob/0a919ce6f278a8fc5c7d203aaa220b3e703bb6b1/source/vendor/vpu/gstreamer_middleware/player/gstreamer_middleware_decode.c#L7

    int change_omx_decode_element_rank()
{
    GstRegistry *registry = gst_registry_get();

    if (!registry) {
        ERROR("Failed to get gstreamer registry");
        return -1;
    }

    char *omx_decode_element_list[] = {        
        "amcviddec-omxqcomvideodecoderh263",
        "amcviddec-omxqcomvideodecoderavc",
        "amcviddec-omxqcomvideodecoderhevc",
        "amcviddec-omxqcomvideodecodermpeg2",
        "amcviddec-omxqcomvideodecodermpeg4",
        "amcviddec-omxqcomvideodecodervp8",
        "amcviddec-omxqcomvideodecodervp9",
        "amcvidenc-omxqcomvideoencoderavc",
        "amcvidenc-omxqcomvideoencoderh263",
        "amcvidenc-omxqcomvideoencoderhevc",
        "amcvidenc-omxqcomvideoencodermpeg4",
        "amcvidenc-omxqcomvideoencodervp8"
    };

    for (int i = 0; i < ARRAYSIZE(omx_decode_element_list); i++) {
        GstPluginFeature *feature =
            gst_registry_lookup_feature(registry, omx_decode_element_list[i]);
        if (!feature) {
            WARN("Featuer does not exist: %s", omx_decode_element_list[i]);
            continue;
        }

        gst_plugin_feature_set_rank(feature, GST_RANK_PRIMARY + 1);
        gst_registry_add_feature(registry, feature);
        gst_object_unref(feature);
    }

    return 0;
}

    Also I noticed that with 1.20.4 incoming HEVC stream should be generated with main-10 profile, since for some reason, main profile is not listed in amcviddec-omxqcomvideodecoderhevc capabilities anymore after gstreamer 1.18.6