ffmpeg, mediacodec hevc decode c/c++

Question

I'm struggling quite a lot of time with mediacodec in ffmpeg and right now I'm not sure if I have achieved expected result or not.

Objective: Use ffmpeg (compiled with mediacodec) in android application from c/c++ library to decode rstp hevc stream.

Right now I'm able to decode the video and it looks like it is working, but im not sure if program is still using CPU or is using HW acceleration (mediacodec). Please have a look at my simplified initialization part:

enum AVPixelFormat getHwFixelFormat(AVCodecContext *ctx, const enum AVPixelFormat *pix_fmts)
{
    return AV_PIX_FMT_MEDIACODEC;
}


AVFormatContext* formatCtx = avformat_alloc_context();
avformat_open_input(&formatCtx, m_link.c_str(), NULL, NULL);
avformat_find_stream_info(&formatCtx, NULL);
avformat_find_stream_info(&formatCtx, NULL);
m_videoStreamIdx = av_find_best_stream(&formatCtx, AVMEDIA_TYPE_VIDEO, -1, -1, &m_decoder, 0);;

const AVCodec* decoderHw = avcodec_find_decoder_by_name("hevc_mediacodec");
m_decoder = decoderHw;

AVCodecContext* decoderCtx = avcodec_alloc_context3(m_decoder);
avcodec_parameters_to_context(decoderCtx, m_decodeParams);

m_hwPixFormat = AV_PIX_FMT_MEDIACODEC;
decoderCtx->get_format = getHwFixelFormat;

AVBufferRef* hwContext = nullptr;
av_hwdevice_ctx_create(&hwContext, m_currHwDecoder->m_devType, nullptr, nullptr, 0);
decoderCtx->hw_device_ctx = av_buffer_ref(hwContext);
avcodec_open2(decoderCtx, m_decoder, nullptr);

And now piece of code responsible for receiving frame:

avcodec_receive_frame(&decoder, frame);

if (frame->format == m_hwPixFormat) {
        /* Branch never reached on android */
    if (av_hwframe_transfer_data(swFrame, frame, 0) < 0) {
        /* Error while copying from hw -> continue reading */
    }
    decodedFrame = &swframeDeleterPtr;
} else {
    decodedFrame = &frameDeleterPtr;
}

Basically I'm not able to meet frame->format == m_hwPixFormat condition and I'm constantly getting AV_PIX_FMT_NV12 instead of AV_PIX_FMT_MEDIACODEC (I'm not sure if thats correct behaviour or not).

Note: I'm using the same code on Windows with other hw acc codecs for example hevc_cuvid and I'm getting AV_PIX_FMT_CUDA as expected and if constraints are met.

-- Tested SW decoding on android -- Funny thing, I used simple SW decoder on android and its working much faster than my current "HW accelerated" solution

Answer 1

To get frames in AV_PIX_FMT_MEDIACODEC formats instead of AV_PIX_FMT_NV12, you should set AVMediaCodecContext->surface (android/view/Surface object reference). https://ffmpeg.org/doxygen/4.0/hwcontext__mediacodec_8h_source.html