I recently trained an object detection model on Google Cloud Vision. I exported the metadat jason file, label text file, and the model tflite file of the trained model and I intend to run it on Android. However, I cannot run this model using the Android demo app as it crashes every time.
The demo app used is compatible with a locally trained and converted tflite model but not the one exported from Google Cloud.
What might be wrong here and how can it be solved?
Thanks
Reference: Demo App: https://github.com/tensorflow/examples/tree/master/lite/examples/object_detection
Partial Log:
2020-01-24 11:29:11.628 18071-18071/org.tensorflow.lite.examples.detection E/libc: Access denied finding property "persist.camera.privapp.list"
2020-01-24 11:29:11.732 18071-18101/org.tensorflow.lite.examples.detection I/tensorflow: CameraConnectionFragment: Opening camera preview: 640x480
2020-01-24 11:29:11.769 18071-18102/org.tensorflow.lite.examples.detection D/vndksupport: Loading /vendor/lib/hw/[email protected] from current namespace instead of sphal namespace.
2020-01-24 11:29:11.770 18071-18102/org.tensorflow.lite.examples.detection D/vndksupport: Loading /vendor/lib/hw/gralloc.msm8937.so from current namespace instead of sphal namespace.
2020-01-24 11:29:11.803 18071-18071/org.tensorflow.lite.examples.detection I/Timeline: Timeline: Activity_idle id: android.os.BinderProxy@5ab1c5e time:332335506
2020-01-24 11:29:12.198 18071-18101/org.tensorflow.lite.examples.detection D/tensorflow: CameraActivity: Initializing buffer 0 at size 307200
2020-01-24 11:29:12.201 18071-18101/org.tensorflow.lite.examples.detection D/tensorflow: CameraActivity: Initializing buffer 1 at size 153599
2020-01-24 11:29:12.203 18071-18101/org.tensorflow.lite.examples.detection D/tensorflow: CameraActivity: Initializing buffer 2 at size 153599
2020-01-24 11:29:12.204 18071-18101/org.tensorflow.lite.examples.detection I/tensorflow: DetectorActivity: Preparing image 1 for detection in bg thread.
2020-01-24 11:29:12.311 18071-18100/org.tensorflow.lite.examples.detection I/tensorflow: DetectorActivity: Running detection on image 1
2020-01-24 11:29:12.475 18071-18100/org.tensorflow.lite.examples.detection E/AndroidRuntime: FATAL EXCEPTION: inference
Process: org.tensorflow.lite.examples.detection, PID: 18071
java.lang.IllegalArgumentException: Cannot convert between a TensorFlowLite buffer with 307200 bytes and a Java Buffer with 270000 bytes.
at org.tensorflow.lite.Tensor.throwIfShapeIsIncompatible(Tensor.java:332)
at org.tensorflow.lite.Tensor.throwIfDataIsIncompatible(Tensor.java:305)
at org.tensorflow.lite.Tensor.setTo(Tensor.java:123)
at org.tensorflow.lite.NativeInterpreterWrapper.run(NativeInterpreterWrapper.java:148)
at org.tensorflow.lite.Interpreter.runForMultipleInputsOutputs(Interpreter.java:296)
at org.tensorflow.lite.examples.detection.tflite.TFLiteObjectDetectionAPIModel.recognizeImage(TFLiteObjectDetectionAPIModel.java:193)
at org.tensorflow.lite.examples.detection.DetectorActivity$2.run(DetectorActivity.java:183)
at android.os.Handler.handleCallback(Handler.java:790)
at android.os.Handler.dispatchMessage(Handler.java:99)
at android.os.Looper.loop(Looper.java:175)
at android.os.HandlerThread.run(HandlerThread.java:65)
=========================================================
Update: Now we know that it is because the image fed to the model and the input shape of the model does not match. The input/output shape of models trained from Google Cloud Vision doesn't seem to be consistent. I recently got one of [ 1 320 320 3] in and [ 1 20 4] out, and another of [ 1 512 512 3] in and [ 1 20 4] out.
The demo app is made to handle models of [ 1 300 300 3] in and [ 1 10 4] out.
How do I assign the shapes of a model before training on Google Cloud Vision or how do I make the demo app capable of handling a model of a specific shape?
=========================================================
As an attempt to enable the demo app to handle a model of a specific shape, I changed TF_OD_API_INPUT_SIZE from 300 to 320, which seems to have solved the input data shape issue. However, problems come at the output side.
The new error log says:
java.lang.IllegalArgumentException: Cannot copy between a TensorFlowLite tensor with shape [1, 20, 4] and a Java object with shape [1, 10, 4].
Changing TEXT_SIZE_DIP from 10 to 20 doesn't help.
The cause for the crash is that the input shape doesn't not match with that of the model, after solving which, another crash is caused due to mismatch of the output shape.
The solution to that is to adjust the I/O shape of the demo application according to the model metadata provided by AutoML on Google Cloud.