Firebase ML Kit : "Internal Error" Exception, but good input & good configuration

Question

Summary

1. Context
2. The problem
3. Trying to solve the problem
4. Question
5. Related questions

Context

I use a custom .tflite model I saved in the Firebase servers (https://firebase.google.com/docs/ml-kit/android/use-custom-models). This is my custom Super-Resolution images GAN generator: I give it a 32x32 image, and it gives me the super resolutioned 128x128 image. I am trying to use it in my Android app.

I followed the documentation whose I gave the link above.

The problem

The following exception is thrown:

I/System.out: com.google.firebase.ml.common.FirebaseMLException: Internal error has occurred when executing Firebase ML tasks

Trying to solve the problem

Analysing the expected input and output shapes

According to the Google Colab Python Interpreter, my expected input and output are the following:

[ 1 32 32 3]

class 'numpy.float32'>

[ 1 128 128 3]

class 'numpy.float32'>

So I must give my generator model a 32x32x3 image, and it must output a 128x128x3 image, everything is normal here :). This is the normal work of a SRGAN generator.

Analysing the Android app sources

Here is the Android app source code... it includes both the configuration and the inference run of my generator model. Data to be sent is also shown.

Configuring the ML Kit client

FirebaseModelDownloadConditions.Builder conditionsBuilder =
        new FirebaseModelDownloadConditions.Builder().requireWifi();
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.N) {
    // Enable advanced conditions on Android Nougat and newer.
    conditionsBuilder = conditionsBuilder
            .requireCharging();
}
FirebaseModelDownloadConditions conditions = conditionsBuilder.build();

cloudSource = new FirebaseRemoteModel.Builder("srgan")
        .enableModelUpdates(true)
        .setInitialDownloadConditions(conditions)
        .setUpdatesDownloadConditions(conditions)
        .build();
FirebaseModelManager.getInstance().registerRemoteModel(cloudSource);

FirebaseModelOptions options = new FirebaseModelOptions.Builder()
        .setRemoteModelName("srgan")
        .build();
FirebaseModelInterpreter firebaseInterpreter =
        FirebaseModelInterpreter.getInstance(options);

FirebaseModelInputOutputOptions inputOutputOptions =
        new FirebaseModelInputOutputOptions.Builder()
                .setInputFormat(0, FirebaseModelDataType.FLOAT32, new int[]{32, 32, 3})
                .setOutputFormat(0, FirebaseModelDataType.FLOAT32, new int[]{128, 128, 3})
                .build();

Defining the data sent as input, a 32x32x3 tensor of the pixels (defines byte[][][] pixels)

Bitmap bitmap = MediaStore.Images.Media.getBitmap(context.getContentResolver(), Uri.parse("file://" + selected_image_uri));
Bitmap bitmapResized = Bitmap.createScaledBitmap(bitmap, 32, 32, false);
ByteArrayOutputStream stream = new ByteArrayOutputStream();
bitmapResized.compress(Bitmap.CompressFormat.PNG, 100, stream);
float[][][] pixels = new float[bitmapResized.getWidth()][bitmapResized.getHeight()][3];
for(int i = 0; i < bitmapResized.getWidth(); i++) {
    for(int j = 0; j < bitmapResized.getHeight(); j++) {
        pixels[i][j][0] = Color.red(bitmapResized.getPixel(i,j));
        pixels[i][j][1] = Color.green(bitmapResized.getPixel(i,j));
        pixels[i][j][2] = Color.blue(bitmapResized.getPixel(i,j));
    }
}

Inference of the generator model (uses the previously defined byte[][][] pixels)

FirebaseModelInputs inputs = new FirebaseModelInputs.Builder()
        .add(pixels)
        .build();
firebaseInterpreter.run(inputs, inputOutputOptions)
        .addOnSuccessListener(
                new OnSuccessListener<FirebaseModelOutputs>() {
                    @Override
                    public void onSuccess(FirebaseModelOutputs result) {
                        SweetAlertDialog pDialog2 = new SweetAlertDialog(context, SweetAlertDialog.SUCCESS_TYPE);
                        pDialog2.setTitleText("GG!");
                        pDialog2.setContentText("Image treated!");
                        pDialog2.show();
                    }
                })
        .addOnFailureListener(
                new OnFailureListener() {
                    @Override
                    public void onFailure(@NonNull Exception e) {
                        System.out.println(e);
                        SweetAlertDialog pDialog2 = new SweetAlertDialog(context, SweetAlertDialog.ERROR_TYPE);
                        pDialog2.setTitleText(context.getResources().getString(R.string.error_firebase_model_interpreter_running));
                        pDialog2.setContentText(context.getResources().getString(R.string.error_firebase_model_interpreter_running_contents));
                        pDialog2.show();
                    }
                });

Question

The thrown exception is very wide. Inputs seem good. Configuration of my client too. What could be wrong? I've no idea at all.

Related questions

• Android MLKit - Internal error has occurred when executing Firebase ML tasks : but my type is FLOAT32, as I configured it: this error of configuration doesn't concern me

• Another SO question where the answer said to check the inputs shape, but mine is rightly 32x32x3.

Answer 1

Problem solved by changing a few lines... stupid error! The key here was to take account of the first dimension, which contains only 1 value (the first "1"). 1 value only since I just send (and receive) only 1 image.

1.

.setInputFormat(0, FirebaseModelDataType.FLOAT32, new int[]{1, 32, 32, 3})


.setOutputFormat(0, FirebaseModelDataType.FLOAT32, new int[]{1, 128, 128, 3})

2. float[][][][] pixels = new float[1][bitmapResized.getWidth()][bitmapResized.getHeight()][3];

3. pixels[0][i][j][0] = Color.red(bitmapResized.getPixel(i,j)); pixels[0][i][j][1] = Color.green(bitmapResized.getPixel(i,j)); pixels[0][i][j][2] = Color.blue(bitmapResized.getPixel(i,j));