AVPacket to Android MediaCodec

Question

I'm using JavaCPP av_read_packet to read AVPackets from H264 (MPEG4 AVC) source, then I'm trying to pass only the video packets to Android's MediaCodec to decode them and to render into a Surface (for display). I've read it's necesary to use av_bitstream_filter_filter with h264_mp4toannexb filter when you use AVC source to be compatible with MediaCodec.

The problem is that dequeueOutputBuffer always return -1, and I don't know if the problem is when use the filter or any other reason.

Here is the code fragment:

bsfc = new AVBitStreamFilterContext();
bsfc = av_bitstream_filter_init("h264_mp4toannexb");

public void FilterPacket(AVPacket paquete){

    IntPointer size_pointer = new IntPointer(1);
    size_pointer.put(paquete.size());
    BytePointer data = new BytePointer();

    int sal = av_bitstream_filter_filter(bsfc, video_codec, (BytePointer)null, data, size_pointer, paquete.data(), paquete.size(), paquete.flags() & AV_PKT_FLAG_KEY );

    paquete.size(size_pointer.get());
    paquete.data(data);

}

And here is what's inside the decode loop:

public void DecoderLoop(AVPacket paquete){
    int inputBufferIndex = decoder.dequeueInputBuffer(5000);
    if (inputBufferIndex >= 0) {
        ByteBuffer bufferSalida = inputBuffers[inputBufferIndex]; 
        FilterPacket(paquete);
        bufferSalida.put(paquete.data().asByteBuffer());
        //double presentationTemp = (double)(paquete.pts()/paquete.duration()) * (1000000/30); 
        //long presentationTime = (long)presentationTemp;
        decoder.queueInputBuffer(inputBufferIndex, 0, paquete.size(), 0, 0);

    }

    int outputBufferIndex = decoder.dequeueOutputBuffer(info, 10000);
    if (outputBufferIndex >= 0) {
        // outputBuffer is ready to be processed or rendered.
        decoder.releaseOutputBuffer(outputBufferIndex, true); 
    } else if (outputBufferIndex == MediaCodec.INFO_OUTPUT_BUFFERS_CHANGED) {
        outputBuffers = decoder.getOutputBuffers();
    } else if (outputBufferIndex == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED) {
        // Subsequent data will conform to new format.
        format = decoder.getOutputFormat();
    }
}

dequeueInputBuffer returns a positive value, and the av_bitstream_filter_filter returns 1 (but i don't know if it's working), but dequeueOutputBuffers return -1 always. Can anyone help me where can be the problem?

I suspect that the problem can be in one of these:

• av_bitstream_filter_filter

• bufferSalida.put(paquete.data().asByteBuffer()); ¿Can i use these way?

• If I use .asByteBuffer, can i use paquete.size() or may I use something different?

Thank you very much

Answer 1

I'm not familiar with this JavaCPP wrapping of the libavcodec libraries, but in addition to the plain AVPackets, you need AVCodecContext.extradata as well (after initializing the bitstream filter, and after filtering the first packet). You need to pass the data from extradata either as a ByteBuffer with the name "csd-0" in the input MediaFormat that you pass to configure, or as a separate buffer with BUFFER_FLAG_CODEC_CONFIG before sending the first AVPacket.