Video frame capture in Android

Question

I'm trying to implement an Android app that captures about 1 picture per second, performs some processing on each picture and sends the output to a file for storage. My first pass at this tries something like the following:

public class MainActivity extends Activity {

    ...
    Handler loopHandler = new Handler();
    Runnable loopRunnable = new Runnable() {

        @Override
        public void run() {
            Thread pictureThread = new Thread(pictureRunnable);
            pictureThread.start();
            loopHandler.postDelayed(this, 1000);
        }
    };

    Runnable pictureRunnable = new Runnable() {
        @Override
        public void run() {
            mCamera.takePicture(null, null, mPicture);
        }    
    };

    private PictureCallback mPicture = new PictureCallback() {
        @Override
        public void onPictureTaken(byte[] data, Camera camera) {
        ... My processing code ...
        }
    }

The app freezes after taking about 4 pictures in this way. So, I'm guessing this is probably too naive an approach but would appreciate a deeper understanding of why this can't work.

Is there any way to do this without engaging with video directly or will I ultimately have to create something that pulls frames out of a video stream?

Answer 1

The best thing to do would be to study your code a bit deeper and understand exactly what is causing the app to freeze. Maybe it's not related to the code you posted but to the actual processing of the image.

Another approach which seemed to work better for me in the past, would be to skip PictureCallback altogether. Instead, you can use PreviewCallback (http://developer.android.com/reference/android/hardware/Camera.PreviewCallback.html). This callback gets triggered on each frame, so you can simply check inside this frame if it's been over 1 second since you last processed an image, and if so, do your image processing on another thread.

I haven't tested this, but something like this:

myCamera.setPreviewCallback(new PreviewCallback() {

    @Override
    public void onPreviewFrame(byte[] data, Camera camera) {

        // previousTime is defined as a member variable
        long timeElapsed = System.currentTimeMillis() - previousTime;

        if(timeElapsed > 1000) {

            // reset values for the next run
            previousTime = System.currentTimeMillis();

            // process the image (just an example, you should do this inside an AsyncTask)
            Size previewSize = myCamera.getParameters().getPreviewSize();
            ByteArrayOutputStream out = new ByteArrayOutputStream();
            YuvImage yuvImage = new YuvImage(data, myCamera.getParameters().getPreviewFormat(), previewSize.width,previewSize.height, null);
            yuvImage.compressToJpeg(new Rect(0, 0, previewSize.width, previewSize.height), 100, out);
            byte[] imageBytes = out.toByteArray();

            Options options = new Options();
            options.inSampleSize = 1;
            Bitmap image = BitmapFactory.decodeByteArray(imageBytes, 0, imageBytes.length, options);

            // you now have the image bitmap which you can use to apply your processing ...
        }

    }
});