IllegalStateException when reading from ShortBuffer in Unity Android

Question

I'm working on a game in Unity, where i need raw audio data from the media buffer. The following code is being used as a jar plugin in Unity. It's working on android 4.x without problems. But when i try to run it on Android 5.0.1 or 5.1, i get the following exception, but only when i try to open aac or m4a files. Any suggestions?

This is the function, where i read the audio buffer:

   public short[] ReadNextSamples() {
        if (sawOutputEOS) {
            return new short[0];
        }
        int res;
        while (true)
        {
            res = codec.dequeueOutputBuffer(info, kTimeOutUs);

        if (res >= 0)
        {
            if (info.size > 0)
            {
                break;
            }
            else
            {
                codec.releaseOutputBuffer(res, false);

                if ((info.flags & MediaCodec.BUFFER_FLAG_END_OF_STREAM)!= 0)
                {
                    sawOutputEOS = true;
                    return new short[0];
                }
            }
        }
    }
    int outputBufIndex = res;
    ByteBuffer buf = codecOutputBuffers[outputBufIndex];
    int sampleSize = info.size / 2;
    short[] sampleValues = new short[sampleSize];
    ShortBuffer shBuf = buf.asShortBuffer();

    try {
        shBuf.get(sampleValues, 0, sampleSize);
    } catch (Exception e) {
        //the exception is being thrown here
        e.printStackTrace();
    }

    codec.releaseOutputBuffer(outputBufIndex, false);
    if ((info.flags & MediaCodec.BUFFER_FLAG_END_OF_STREAM) != 0) {
        sawOutputEOS = true;
    }
    return sampleValues;
}

The exception:

06-04 16:18:01.188: W/System.err(20884): java.lang.IllegalStateException: buffer is inaccessible
06-04 16:18:01.188: W/System.err(20884): at java.nio.DirectByteBuffer.checkIsAccessible(DirectByteBuffer.java:551)
06-04 16:18:01.188: W/System.err(20884): at java.nio.DirectByteBuffer.get(DirectByteBuffer.java:155)
06-04 16:18:01.188: W/System.err(20884): at java.nio.ByteBufferAsShortBuffer.get(ByteBufferAsShortBuffer.java:103)
06-04 16:18:01.190: W/System.err(20884): at com.pocketgames.musiverse.MediaDecoder.ReadNextSamples(MediaDecoder.java:112)
06-04 16:18:01.190: W/System.err(20884): at com.unity3d.player.ReflectionHelper.nativeProxyInvoke(Native Method)
06-04 16:18:01.190: W/System.err(20884): at com.unity3d.player.ReflectionHelper.a(Unknown Source)
06-04 16:18:01.190: W/System.err(20884): at com.unity3d.player.ReflectionHelper$1.invoke(Unknown Source)
06-04 16:18:01.190: W/System.err(20884): at java.lang.reflect.Proxy.invoke(Proxy.java:397)
06-04 16:18:01.190: W/System.err(20884): at $Proxy0.run(Unknown Source)
06-04 16:18:01.191: W/System.err(20884): at java.lang.Thread.run(Thread.java:818)

Answer 1

Basically you should use the API level dependent code. I think you might have already figured this out. But for the sake of documenting, you should use

ByteBuffer buf = codecOutputBuffers[outputBufIndex];

for API level < 21

and

ByteBuffer buf = codec.getOutputBuffer(outputBufIndex);

for API level >= 21.

The code would look something like this:

final int version = Build.VERSION.SDK_INT;
ByteBuffer buf;
if (version >= 21) {
    buf = codec.getOutputBuffer(outputBufIndex);
} else {
    buf = codecOutputBuffers[outputBufIndex];
}

If you look at the source code of MediaCodec class, the code for getOutputBuffer is:

@Nullable
public ByteBuffer getOutputBuffer(int index) {
    ByteBuffer newBuffer = getBuffer(false /* input */, index);
    synchronized(mBufferLock) {
        invalidateByteBuffer(mCachedOutputBuffers, index);
        mDequeuedOutputBuffers.put(index, newBuffer);
    }
    return newBuffer;
}

So, the error that you get is related to acquiring a lock mBufferLock which might be required with the implementation of MediaCodec for API level >= 21 and hence you get the error related to inaccessibility. I hope this helps.