ARCore and TFlite GPUDelegate Availability

Question

Quick Summary

After I added ARCore to my Android Application, I can no longer use the GPU Delegate from the TFLite API to run my ML Model on the phones GPU.

I am unaware of a way to get back GPU access without ditching ARCore from my application.

My Devices

I have tested this on Google Pixel 7 Pro and Google Pixel Tablet Devices, both of which are supported by ARCore and provide GPU access to TFLite.

Background

I am working on an Android Application that uses both ARCore and TFLite. Before adding ARCore, I got the TFLite Model to work with the Interpreter API.

I check for availability of the GPU Delegate with CompatibilityList().isDelegateSupportedOnThisDevice, only adding it if it is available.

val interpreterOptions = Interpreter.Options().apply {
    addDelegate(NnApiDelegate())
    if (CompatibilityList().isDelegateSupportedOnThisDevice) {
        addDelegate(GpuDelegate(CompatibilityList().bestOptionsForThisDevice))
    } else {
        Log.w("MyApp", "GPU Delegate is not supported on this device.")
    }
}

and after loading my model, I am instantiate the interpreter from these options.

val interpreter = Interpreter(myModel, interpreterOptions)

However, after adding ARCore to my application,I seem have lost access to the GPU of my device (i.e. isDelegateSupportedOnThisDevice == false always).

Aside from the permissions, I have added the following statements to my AndroidManifest.xml.

<uses-feature android:name="android.hardware.camera.ar" />
<uses-feature android:name="com.google.ar.core.depth" />

<application ...>
    <!-- require ARCore (https://developers.google.com/ar/develop/java/enable-arcore) -->
    <meta-data
        android:name="com.google.ar.core"
        android:value="required" />
    <!-- other stuff ... -->
</application>

and in build.gradle.kts

dependencies {
    // ARCore
    implementation("com.google.ar:core:1.33.0")
    implementation("de.javagl:obj:0.2.1") // OBJ file loader (for compatibility purposes only!)
    // ... other stuff ...
}

And implemented ARCore using the OpenGL Wrapper Code based on the Hello AR Kotlin Example.

However, I quickly noticed that my model now does not run nearly as fast, and debugging revealed that the GPU Delegate is now never available on any device that I run ARCore on. But just deactivating ARCore does not suffice, I need to remove all of the above dependency statements for GPU Delegate access to work again.

Troubleshooting I tried

Remove ARCore references from the MainActivity class

(i.e. don't use ARCore but keep the dependencies) - does not change anything

Remove ARCore references AND dependencies from the entire project

GPU access now works again

Declare explicit usage of OpenGL and OpenCL

I stumbled on this GitHub issue in the TensorFlow which talks about GPU Delegate not being accessible on third party phones. They do not use ARCore but experience somewhat similar issues.

It says to add the following statements to my <activity/> in AndroidManifest.xml.

<uses-library
    android:name="libOpenCL.so"
    android:required="false" />
<uses-library
    android:name="libOpenCL-pixel.so"
    android:required="false" />

which did not change anything.

I still strongly believe that using ARCore is the cause of this issue. I am also aware that ARCore does at least some processing on the GPU (hence the OpenGL wrappers), but I have not found any issues regarding this specific problem with the TFLite API.

Is there a way to gain back access to the GPU Delegate?

Edit - Regarding ARCore + ML Sample

Maybe it is important to mention that although there is an Example of using ARCore with ML available, this uses Googles MLKit which does provide wrappers for High Level ML Tasks, some of which have built-in GPU support.

However, my model does not fall into the range of application covered by these tasks, so the only other option would be to fallback to TFLite, which is basically what I am doing right now.

Moreover, the MLKit API does not expose GPU settings as far as I can tell.

Answer 1

Solution

I moved all TFLite implementations to use Google Play Services (in favor of using org.tensorflow).

implementation("com.google.android.gms:play-services-tflite-impl:16.1.0")
implementation("com.google.android.gms:play-services-tflite-java:16.1.0")
implementation("com.google.android.gms:play-services-tflite-support:16.1.0")
implementation("com.google.android.gms:play-services-tflite-gpu:16.2.0")

Note that this also required me to use a slightly different Interpreter API.

The Issue

So removing all the ARCore dependencies was not what did the trick exactly. I did this by reverting via git and in the process it seems I also mixed up TFLite dependencies from different sources. Namely, I used TFLite from Maven

implementation("org.tensorflow:tensorflow-lite-task-vision:0.4.0")
implementation("org.tensorflow:tensorflow-lite:2.9.0")
implementation("org.tensorflow:tensorflow-lite-gpu-delegate-plugin:0.4.0")
implementation("org.tensorflow:tensorflow-lite-gpu:2.9.0")

togeher with some packages from Google Play Services

implementation("com.google.android.gms:play-services-tflite-impl:16.1.0")
implementation("com.google.android.gms:play-services-tflite-gpu:16.2.0")

For more information, see Googles Documentation on GPU acceleration delegate with Interpreter API

and even though I made no changes to how I use the TFLite API, this seemed to be enough to break compatibility.

I am unsure how exactly I got this mixed up, but I figure it's because different parts of the TFLite Examples and Documentation use different APIs and this distinction slipped past me.