I'm trying to decode a simple H.264 video stream using MediaConfig on Android. It's just throwing an exception on the first packet and for the life of me I can't understand why.
Here's the minimal code. All it does is get a codec, start the decoder, grab an input buffer, fill it and submit it. But on a debug device, I get an UnsupportedOperationException (also below).
This has been driving me bananas, not helped by spending a day on the emulator before realising that it wasn't doing any video decoding at all.
Code (Stripped of just about everything except the problem):
package com.AndroidH264VideoTest;
import android.app.Activity;
import android.os.Bundle;
import android.media.MediaCodec;
import android.media.MediaFormat;
import java.nio.ByteBuffer;
public class AndroidH264VideoTest extends Activity
{
private static final int SHORT_TIMEOUT = 1; // Provide some timeout to getting a buffer
private MediaCodec m_decoder = null;
private ByteBuffer[] m_decoderInputBuffers;
@Override
public void onCreate(Bundle savedInstanceState)
{
super.onCreate(savedInstanceState);
String MIME_TYPE = "video/avc"; // This is the type for H.264
int TEST_WIDTH = 8; // Our test image is 8x8 pixels
int TEST_HEIGHT = 8;
m_decoder = MediaCodec.createDecoderByType(MIME_TYPE);
MediaFormat decoderFormat = MediaFormat.createVideoFormat(MIME_TYPE, TEST_WIDTH, TEST_HEIGHT);
m_decoder.configure(decoderFormat, null, null, 0); // bytebuffer input/output
m_decoder.start();
m_decoderInputBuffers = m_decoder.getInputBuffers();
}
@Override
public void onResume()
{
byte SPS_PACKET[] =
{
(byte)0x00, (byte)0x00, (byte)0x00, (byte)0x01, // NAL header identifier
(byte)0x67, // NAL info - (byte)0x67 = 0 11 00111 = type 7 = SPS (1st packet)
(byte)0x42, // SPS profile_idc = (byte)0x42/66 - Baseline. Should be decodable by anything
(byte)0xC0, // Constrained - should be even more decodable.
(byte)0x0A, // SPS level_idc = (byte)0x0a/10
(byte)0xDA, (byte)0x7E, (byte)0x59, (byte)0x66, (byte)0xA0, (byte)0xC0, (byte)0x20,
(byte)0xC8, (byte)0x00, (byte)0x00, (byte)0x03, (byte)0x00, (byte)0x08, (byte)0x00, (byte)0x00,
(byte)0x03, (byte)0x03, (byte)0xc4, (byte)0x78, (byte)0x91, (byte)0x35,
};
super.onResume(); // Always call the superclass method first
// Take ownership of buffer
int inputBufferId = m_decoder.dequeueInputBuffer(SHORT_TIMEOUT);
if (inputBufferId < 0)
{
// No buffer available error
return;
}
// Copy data to buffer
m_decoderInputBuffers[inputBufferId] = ByteBuffer.wrap(SPS_PACKET);
// Submit buffer for processing
m_decoder.queueInputBuffer(inputBufferId, 0, SPS_PACKET.length, 0, MediaCodec.BUFFER_FLAG_SYNC_FRAME);
}
}
So, that should be harmless. Won't do anything because I've only passed one NAL packet, but it shouldn't complain.
However, on my Marshmallow test device this happens:
8-21 16:21:45.295 21871 21871 V ActivityThread: Performing resume of ActivityRecord{fa15f01 token=android.os.BinderProxy@e6df1a6 {com.AndroidH264VideoTest.AndroidH264VideoTest/com.AndroidH264VideoTest.AndroidH264VideoTest}}
08-21 16:21:45.296 21871 21871 D AndroidRuntime: Shutting down VM
--------- beginning of crash
08-21 16:21:45.297 21871 21871 E AndroidRuntime: FATAL EXCEPTION: main
08-21 16:21:45.297 21871 21871 E AndroidRuntime: Process: com.AndroidH264VideoTest.AndroidH264VideoTest, PID: 21871
08-21 16:21:45.297 21871 21871 E AndroidRuntime: java.lang.RuntimeException: Unable to resume activity {com.AndroidH264VideoTest.AndroidH264VideoTest/com.AndroidH264VideoTest.AndroidH264VideoTest}: java.lang.UnsupportedOperationException
08-21 16:21:45.297 21871 21871 E AndroidRuntime: at android.app.ActivityThread.performResumeActivity(ActivityThread.java:3325)
08-21 16:21:45.297 21871 21871 E AndroidRuntime: at android.app.ActivityThread.handleResumeActivity(ActivityThread.java:3356)
08-21 16:21:45.297 21871 21871 E AndroidRuntime: at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:2670)
08-21 16:21:45.297 21871 21871 E AndroidRuntime: at android.app.ActivityThread.-wrap11(ActivityThread.java)
08-21 16:21:45.297 21871 21871 E AndroidRuntime: at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1499)
08-21 16:21:45.297 21871 21871 E AndroidRuntime: at android.os.Handler.dispatchMessage(Handler.java:111)
08-21 16:21:45.297 21871 21871 E AndroidRuntime: at android.os.Looper.loop(Looper.java:207)
08-21 16:21:45.297 21871 21871 E AndroidRuntime: at android.app.ActivityThread.main(ActivityThread.java:5765)
08-21 16:21:45.297 21871 21871 E AndroidRuntime: at java.lang.reflect.Method.invoke(Native Method)
08-21 16:21:45.297 21871 21871 E AndroidRuntime: at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:789)
08-21 16:21:45.297 21871 21871 E AndroidRuntime: at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:679)
08-21 16:21:45.297 21871 21871 E AndroidRuntime: Caused by: java.lang.UnsupportedOperationException
08-21 16:21:45.297 21871 21871 E AndroidRuntime: at java.nio.ByteBuffer.setAccessible(ByteBuffer.java:636)
08-21 16:21:45.297 21871 21871 E AndroidRuntime: at android.media.MediaCodec.invalidateByteBuffer(MediaCodec.java:2646)
08-21 16:21:45.297 21871 21871 E AndroidRuntime: at android.media.MediaCodec.queueInputBuffer(MediaCodec.java:2174)
08-21 16:21:45.297 21871 21871 E AndroidRuntime: at com.AndroidH264VideoTest.AndroidH264VideoTest.onResume(AndroidH264VideoTest.java:68)
08-21 16:21:45.297 21871 21871 E AndroidRuntime: at android.app.Instrumentation.callActivityOnResume(Instrumentation.java:1268)
08-21 16:21:45.297 21871 21871 E AndroidRuntime: at android.app.Activity.performResume(Activity.java:6392)
08-21 16:21:45.297 21871 21871 E AndroidRuntime: at android.app.ActivityThread.performResumeActivity(ActivityThread.java:3310)
08-21 16:21:45.297 21871 21871 E AndroidRuntime: ... 10 more
So, why an unsupported operation in queueInputBuffer()?
Update: Oddly on a different device (Android 4.4.4 KitKat, S4 mini) I get a different error message but it still blows up:
E/AndroidRuntime(25080): FATAL EXCEPTION: main
E/AndroidRuntime(25080): Process: com.AndroidH264VideoTest.AndroidH264VideoTest, PID: 25080
E/AndroidRuntime(25080): java.lang.RuntimeException: Unable to start activity ComponentInfo{com.AndroidH264VideoTest.AndroidH264VideoTest/com.AndroidH264VideoTest.AndroidH264VideoTest}: java.lang.IllegalStateException
E/AndroidRuntime(25080): at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2548)
E/AndroidRuntime(25080): at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:2607)
E/AndroidRuntime(25080): at android.app.ActivityThread.access$900(ActivityThread.java:174)
E/AndroidRuntime(25080): at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1325)
E/AndroidRuntime(25080): at android.os.Handler.dispatchMessage(Handler.java:102)
E/AndroidRuntime(25080): at android.os.Looper.loop(Looper.java:146)
E/AndroidRuntime(25080): at android.app.ActivityThread.main(ActivityThread.java:5756)
E/AndroidRuntime(25080): at java.lang.reflect.Method.invokeNative(Native Method)
E/AndroidRuntime(25080): at java.lang.reflect.Method.invoke(Method.java:515)
E/AndroidRuntime(25080): at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:1291)
E/AndroidRuntime(25080): at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1107)
E/AndroidRuntime(25080): at dalvik.system.NativeStart.main(Native Method)
E/AndroidRuntime(25080): Caused by: java.lang.IllegalStateException
E/AndroidRuntime(25080): at android.media.MediaCodec.native_configure(Native Method)
E/AndroidRuntime(25080): at android.media.MediaCodec.configure(MediaCodec.java:262)
E/AndroidRuntime(25080): at com.AndroidH264VideoTest.AndroidH264VideoTest.onCreate(AndroidH264VideoTest.java:33)
E/AndroidRuntime(25080): at android.app.Activity.performCreate(Activity.java:5619)
E/AndroidRuntime(25080): at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1093)
E/AndroidRuntime(25080): at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2512)
E/AndroidRuntime(25080): ... 11 more
The problem on the 4.4.4 was trying to decode an 8x8 image, which was just too small for its tiny brain. If I replace 8x8 with 320x240, or 160x160, or even 67x71 it's happy as long as it's 64x64 or larger.
I'll have to see if this arbitrary size restriction also applies to the Marshmallow device.