Android: mobilenet_v1_1.0_224.tflite model doesn't return bounding box information

Question

I am using https://github.com/anupamchugh/AndroidTfLiteCameraX source code to learn about integrating the TFLite model with Android. I have labels.txt with all the classes in the Assets folder as well.

Currently, this project returns only the prediction class. I want to retrieve the bounding box along with it.

   private fun getMaxResult(result: FloatArray): Int {
        var probability = result[0]
        var index = 0
        for (i in result.indices) {
            if (probability < result[i]) {
                probability = result[i]
                index = i
            }
        }
        return index
    }


    private fun classify(bitmap: Bitmap): String {

        check(isInitialized) { "TF Lite Interpreter is not initialized yet." }
        val resizedImage =
            Bitmap.createScaledBitmap(bitmap, inputImageWidth, inputImageHeight, true)

        val byteBuffer = convertBitmapToByteBuffer(resizedImage)

        val output = Array(1) { FloatArray(labels.size) }
        val startTime = SystemClock.uptimeMillis()
        interpreter?.run(byteBuffer, output)
        val endTime = SystemClock.uptimeMillis()

        var inferenceTime = endTime - startTime
        var index = getMaxResult(output[0])
        var result = "Prediction is ${labels[index]}\nInference Time ${inferenceTime}\""

        return result
    }

This is where the classification occurs. I don't think it returns the bounding box information, but I'm not sure. I don't have much knowledge of tflite model. To get the bounding box, do I need to use a different model? What do I do?

Answer 1

The source code and model you linked is for Image Classification not Object Detection. So there are no bounding boxes coordinatioon / information.

Here is my example code for CameraX and Object Detection https://github.com/FelixAhrens/CameraX-ImageAnalysis-Beta06-Java-TFLite-ObjectDetection

But it's java. For Kotlin you should search explizit for Object Detection and Tensorflow or maybe MLKit (it's in some way easier to implement, also you can use classification models for detection, because the api search for objects in the image by themself. check https://developers.google.com/ml-kit/vision/object-detection and here is a tutorial which is maybe helpful).