I have a SurfaceView that I use as Display for MediaPlayer. I also need to alternatively draw on the same Surface manually by locking/unlocking the Canvas. This never happens at the same time though!
The problem is that as soon I have once locked and unlocked the Surface, it will be unusable for MediaPlayer. The other way round is no problem. I can play a video on the Surface, reset the player and then manually draw on the Surface as I like.
This works:
This fails:
It will fail with an IllegalStateException when MediaPlayer.prepare() is called. Strangely, the message of the exception is "null".
The same happens with a Surface that is used in a SurfaceTexture (e.g. in a GL context).
I wonder what happens with the Surface after locking/unlocking it, that MediaPlayer does not like it anymore.
I might need to end up using multiple Surfaces and switching between them. But I'd like to avoid that since I have a setup with a mix of multiple SurfaceViews and also GlSurfaceViews using SurfaceTextures.
Here is some code to reproduce the phenomenon. Make sure that VIDEO_FILE points to a valid video.
MainActivity.java:
import android.app.Activity;
import android.graphics.Canvas;
import android.media.MediaPlayer;
import android.os.Bundle;
import android.util.Log;
import android.view.Surface;
import android.view.SurfaceHolder;
import android.view.SurfaceView;
import android.view.View;
import android.widget.Button;
public class MainActivity extends Activity {
// Make sure this points to a valid video
private String VIDEO_FILE = "/sdcard/Movies/some-video.mp4";
private MediaPlayer mediaPlayer;
private SurfaceView surfaceView;
private SurfaceHolder surfaceHolder;
private Button playButton;
private Button stopButton;
private Button drawButton;
private boolean alternate;
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
mediaPlayer = new MediaPlayer();
surfaceView = (SurfaceView)findViewById(R.id.video_surfaceview);
surfaceHolder = surfaceView.getHolder();
playButton = (Button)findViewById(R.id.play_button);
stopButton = (Button)findViewById(R.id.stop_button);
drawButton = (Button)findViewById(R.id.draw_button);
surfaceHolder.addCallback(new SurfaceHolder.Callback() {
@Override
public void surfaceCreated(SurfaceHolder surfaceHolder) {
mediaPlayer.setSurface(surfaceHolder.getSurface());
// Make sure we only do things when surface is ready
playButton.setEnabled(true);
stopButton.setEnabled(true);
drawButton.setEnabled(true);
}
@Override
public void surfaceChanged(SurfaceHolder surfaceHolder, int i, int i1, int i2) {
}
@Override
public void surfaceDestroyed(SurfaceHolder surfaceHolder) {
}
});
playButton.setOnClickListener(new View.OnClickListener() {
@Override
public void onClick(View view) {
mediaPlayer.setDisplay(surfaceHolder);
try {
mediaPlayer.setDataSource(VIDEO_FILE);
mediaPlayer.prepare();
mediaPlayer.start();
} catch (Exception e) {
Log.e("Main", "Play failed: " + e.getMessage(), e);
}
}
});
stopButton.setOnClickListener(new View.OnClickListener() {
@Override
public void onClick(View view) {
mediaPlayer.reset();
}
});
drawButton.setOnClickListener(new View.OnClickListener() {
@Override
public void onClick(View view) {
Surface surface = surfaceHolder.getSurface();
Canvas canvas = surfaceHolder.lockCanvas(null);
// Toggle between green and red
if (alternate) {
canvas.drawARGB(255, 255, 0, 0);
} else {
canvas.drawARGB(255, 0, 255, 0);
}
alternate = !alternate;
surfaceHolder.unlockCanvasAndPost(canvas);
}
});
}
}
layout/main_activity.xml:
<?xml version="1.0" encoding="utf-8"?>
<LinearLayout
xmlns:android="http://schemas.android.com/apk/res/android"
android:orientation="vertical"
android:layout_width="match_parent"
android:layout_height="match_parent">
<SurfaceView xmlns:android="http://schemas.android.com/apk/res/android"
android:id="@+id/video_surfaceview"
android:layout_width="fill_parent"
android:layout_height="300dp" />
<Button
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:id="@+id/play_button"
android:enabled="false"
android:text="Play"></Button>
<Button
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:id="@+id/stop_button"
android:enabled="false"
android:text="Stop"></Button>
<Button
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:id="@+id/draw_button"
android:enabled="false"
android:text="Draw"></Button>
</LinearLayout>
There is three buttons: "Play", "Stop", "Draw". You may press "Play" and "Stop" several times (which should work as expected). Then press "Draw" multiple times (this should draw alternating red and green). Now press "Play" again (this will fail).
If you start with "Draw" in the beginning, then "Play" will never work.
Make sure to press "Stop" before pressing "Draw" when a video is playing. Otherwise it will crash (for a valid reason).
I just was wondering whether I should be proud of that Tumbleweed badge that I got for this question and decided to do one last research on that topic. The first thing I then stumbled across was this architecture documentation about Surface and SurfaceHolder:
https://source.android.com/devices/graphics/arch-sh
It's funny, because it seems to contain the exact answer of my question:
When you lock a Surface for Canvas access, the "CPU renderer" connects to the producer side of the BufferQueue and does not disconnect until the Surface is destroyed. Most other producers (like GLES) can be disconnected and reconnected to a Surface, but the Canvas-based "CPU renderer" cannot. This means you can't draw on a surface with GLES or send it frames from a video decoder if you've ever locked it for a Canvas.
Meanwhile I have implemented a solution with multiple surfaces anyhow. I suppose I will now stick to it.
Thanks for listening.