Search code examples
javaandroidbitmapobject-detection

How to create a bitmap whose dimension is the central square of the screen?


Context & Goal

I'm building a real-time object detection app. To achieve this, I need a bitmap to send to the recognition algorithm. I have an activity displaying the camera output in full screen, but I'm trying to operate the recognition only on the central square of the screen. So I need a bitmap with those dimensions and position, corresponding to the the central square of the phone screen.

What I have tried

    @Override
    public void onPreviewSizeChosen(final Size size, final int rotation) {

        /* some code... */
    
        // Width of the screen (larger than the height), for my phone it's 4032
        previewWidth    = size.getWidth();
        // Height of the screen, for my phone it's 1980
        previewHeight   = size.getHeight();

        sensorOrientation = rotation - getScreenOrientation();

        // The bitmap which will be used to the recognition, dimension : full screen of the phone
        rgbFrameBitmap  = Bitmap.createBitmap(previewWidth, previewHeight, Config.ARGB_8888);

        // squared bitmap 320x320, required size for the recognition algorithm
        croppedBitmap   = Bitmap.createBitmap(cropSize, cropSize, Config.ARGB_8888);

        // bitmap to croppedBitmap transform Matrix
        frameToCropTransform =
                ImageUtils.getTransformationMatrix(
                        previewWidth, previewHeight,
                        cropSize, cropSize,
                        sensorOrientation, MAINTAIN_ASPECT);

        cropToFrameTransform = new Matrix();
        frameToCropTransform.invert(cropToFrameTransform);

        trackingOverlay = (OverlayView) findViewById(R.id.tracking_overlay);
        trackingOverlay.addCallback(
                new DrawCallback() {
                    @Override
                    public void drawCallback(final Canvas canvas) {
                        tracker.draw(canvas, isDebug());
                    }
                });
        tracker.setFrameConfiguration(previewWidth, previewHeight, sensorOrientation);
       
    }

@Override
    protected void processImage() {

        trackingOverlay.postInvalidate();

        if (computingDetection) {
            readyForNextImage();
            return;
        }
        computingDetection = true;

        // THE LINE I CAN'T FIGURE OUT
        // I try to set width and height = previewHeight (1980 for my phone so a square)
        // and to move it down by (previewWidth-previewHeight)/2 so it goes in the middle of the screen
        rgbFrameBitmap.setPixels(getRgbBytes(), 0, previewWidth, 0, (previewWidth-previewHeight)/2, previewHeight, previewHeight);

        readyForNextImage();

        final Canvas canvas = new Canvas(croppedBitmap);
        canvas.drawBitmap(rgbFrameBitmap, frameToCropTransform, null);

        // Saved set to true, so I can check what the bitmap sent to the recognition algorithm looks like
        if (SAVE_PREVIEW_BITMAP) {
            ImageUtils.saveBitmap(croppedBitmap);
        }

        runInBackground(
                new Runnable() {
                    @Override
                    public void run() {
                        // Here we send the bitmap to the recognition algorithm to perform real-time detection
                        final List<Classifier.Recognition> results = detector.recognizeImage(croppedBitmap);


/* some code ... */

}

If I replace the (previewWidth-previewHeight)/2 by 0 as the y parameter of rgbFrameBitmap.setPixels() function, I manage to perform real-time recognition in a square on top of the screen, as I want unless it's not centered. But as soon as I set the y parameter to (previewWidth-previewHeight)/2, this is the error I get when it tries to save the bitmap :

W/ImageReader_JNI: Unable to acquire a buffer item, very likely client tried to acquire more than maxImages buffers

How can I get the bitmap to be centered in the middle of the screen ?


Solution

  • Self solved

    There are two thing I misunderstood on the above code.

    First, I mixed up the x and the y parameters of the rgbFrameBitmap.setPixels() function, but in any case this was just adding some black space to the bitmap. To obtain the center of the screen, the parameter to change is the offset, like this :

    int x = (previewWidth-previewHeight)/2;
    rgbFrameBitmap.setPixels(getRgbBytes(), x, previewWidth, 0, 0, previewHeight, previewHeight);
    

    Besides, as we send the croppedBitmap to the recognition algorithm, which is the rgbFrameBitmap cropped to 320x320 thanks to the matrix frameToCropTransform, I had to update this :

    frameToCropTransform =
                    ImageUtils.getTransformationMatrix(
                            previewWidth, previewHeight,
                            cropSize, cropSize,
                            sensorOrientation, MAINTAIN_ASPECT);
    

    into this :

    frameToCropTransform =
                    ImageUtils.getTransformationMatrix(
                            previewHeight, previewHeight,
                            cropSize, cropSize,
                            sensorOrientation, MAINTAIN_ASPECT);
    

    as rgbFrameBitmap is now a square of dimensions previewHeightxpreviewHeight