Search code examples
androidbitmapcamerasurfaceview

Creating processable buffer of previewed camera image

I need to prepare a vision system sample on Android - image processing stuff. There are many samples featuring preview from camera, ex. Sample

The problem is I can't get access to the image, so I can process the image or draw on it in between camera capture and preview.

How to export preview into bitmap or anything processable on single pixel level? How to export it back on screen to debug?

I have tried to modify code in the sample, to change surface view into bitmap and modify it. Unfortunately I have no idea how to return back to surface view:

    public CameraPreview (Context context)
        : base (context)
    {
        surfaceView = new SurfaceView (context);


        windowManager = Context.GetSystemService (Context.WindowService).JavaCast<IWindowManager> ();

        IsPreviewing = false;

        Bitmap drawingCache = surfaceView.DrawingCache;
        Bitmap bitmap = Bitmap.CreateBitmap(drawingCache);

        for(int n = 0; n < 100; n++) {bitmap.SetPixel(n,n,Color.Black); } //I created this for debugging purposes, just to check if everything works fine

        holder = surfaceView.Holder;

        holder.AddCallback (this);
        surfaceView = // HERE I HAVE TO CONVERT BITMAP INTO SURFACE

        AddView (surfaceView);
    }

Here it is all in Xamarin, but I've checked also examples on android arsenal and android developers websites - this sample is just the most simple one I found.

regards

Solution

  • The supported flow on Android is that live preview is separate from the the camera data callback that provides pixels for your image processing.

    If you use TextureSurface, you can apply any OpenGL shader to manipulate the live preview, but this is limited; passing pixels from OpenGL to CPU is possible, but too slow to keep up with acceptable live preview FPS.

    You can use OpenCV, and put your image processing code in the onCameraFrame() callback. The result of the callback (RGB) will be used for live preview. The FPS is lower than what you can get on SurfaceView, but probably best that you can get.