Search code examples
androidfirebase-mlkitandroid-camerax

Translate Firebase MLKit Bounding box coordinates to screen view coordinates

I am using the FirebaseVision Object detection to detect things from the CameraX camera preview. It is detecting things find but I am trying to draw the bounding box of the items detected over the camera preview. In doing so the bounding box that firebase gives back is not for the image itself not the preview view to they appear in the wrong place.

The image size that I get back from firebase is 1200x1600 and the preview size is 2425x1440

How do I translate the bounding boxes returned from firebase to the correct screen coordinates?

Solution

  • What I ended up doing was I took the image size that the camera took, divided the width/height by the view width/height to get the scale size

    if(isPortraitMode()){
    _scaleY = overlayView.height.toFloat() / imageWidth.toFloat()
    _scaleX = overlayView.width.toFloat() / imageHeight.toFloat()
}else{
    _scaleY = overlayView.height.toFloat() / imageHeight.toFloat()
    _scaleX = overlayView.width.toFloat() / imageWidth.toFloat()
}

    Now that I have the scale I can then take the bounding box return by the firebase detector and translate the x and y coordinates by the scales

    private fun translateX(x: Float): Float = x * _scaleX
private fun translateY(y: Float): Float = y * _scaleY

private fun translateRect(rect: Rect) = RectF(
    translateX(rect.left.toFloat()),
    translateY(rect.top.toFloat()),
    translateX(rect.right.toFloat()),
    translateY(rect.bottom.toFloat())
)

    Which then gives you the scaled rect coordinates which you then draw on the screen