How to draw a mark on a camera stream with Xamarin.Android?

Question

I managed to display the camera stream on a TextureView, and to make a Bitmap to read the pixels each frame of the stream. Then I can apply a tracking algorithm to follow a ball for example. But now, I would like to display a red mark (for instance) which shows the position of the found ball. How can I do that?

Answer 1

I believe that there are many ways to draw a mark on the camera view. Here I used a SurfaceView. Basic idea is to set a SurfaceView on the top of Textureview with a transparent background.

<RelativeLayout xmlns:android="http://schemas.android.com/apk/res/android"
    android:layout_width="match_parent"
    android:layout_height="match_parent">

  <TextureView android:id="@+id/textureView"
               android:layout_height="match_parent"
               android:layout_width="match_parent" />

  <SurfaceView
        android:id="@+id/surfaceview"
        android:layout_height="match_parent"
        android:layout_width="match_parent" />
</RelativeLayout >

code behind:

public class MainActivity : Activity, TextureView.ISurfaceTextureListener
{
    private Android.Hardware.Camera _camera;
    private TextureView _textureView;
    private SurfaceView _surfaceView;
    private ISurfaceHolder holder;

    protected override void OnCreate(Bundle bundle)
    {
        base.OnCreate(bundle);

        // Set our view from the "main" layout resource
        SetContentView(Resource.Layout.Main);

        _textureView = (TextureView)FindViewById(Resource.Id.textureView);
        _textureView.SurfaceTextureListener = this;

        _surfaceView = (SurfaceView)FindViewById(Resource.Id.surfaceview);
        //set to top layer
        _surfaceView.SetZOrderOnTop(true);
        //set the background to transparent
        _surfaceView.Holder.SetFormat(Format.Transparent);
        holder = _surfaceView.Holder;
        _surfaceView.Touch += _surfaceView_Touch;
    }

    private void _surfaceView_Touch(object sender, View.TouchEventArgs e)
    {
        //define the paintbrush
        Paint mpaint = new Paint();
        mpaint.Color = Color.Red;
        mpaint.SetStyle(Paint.Style.Stroke);
        mpaint.StrokeWidth = 2f;

        //draw
        Canvas canvas = holder.LockCanvas();
        //clear the paint of last time
        canvas.DrawColor(Color.Transparent, PorterDuff.Mode.Clear);
        //draw a new one, set your ball's position to the rect here
        var x = e.Event.GetX();
        var y = e.Event.GetY();
        Rect r = new Rect((int)x, (int)y, (int)x + 100, (int)y + 100);
        canvas.DrawRect(r, mpaint);
        holder.UnlockCanvasAndPost(canvas);
    }

    public bool OnSurfaceTextureDestroyed(SurfaceTexture surface)
    {
        _camera.StopPreview();
        _camera.Release();

        return true;
    }

    public void OnSurfaceTextureAvailable(SurfaceTexture surface, int width, int height)
    {
        _camera = Android.Hardware.Camera.Open();

        try
        {
            _camera.SetPreviewTexture(surface);
            _camera.SetDisplayOrientation(90);
            _camera.StartPreview();
        }
        catch (Java.IO.IOException ex)
        {
            Console.WriteLine(ex.Message);
        }
    }

    public void OnSurfaceTextureSizeChanged(SurfaceTexture surface, int width, int height)
    {
    }

    public void OnSurfaceTextureUpdated(SurfaceTexture surface)
    {
    }
}

Since I don't have a tracking algorithm to follow a ball, I used the Touch event of the SurfaceView, each time when you tapped on the SurfaceView, a rectangle with red stroke will be drawn on the tapped position. You can modify the code in my _surfaceView_Touch method to your tracking algorithm and draw your red mark.