Search code examples
androidkotlinaccessibilityandroid-servicegesture

What is causing dispatchGesture to fail?


I have an accessibility service defined as:

class ClickService : AccessibilityService() {
    // Pretend the necessary methods are overrided

    override fun onCreate() {
        super.onCreate()
        val binding = TODO() // Pretend we have a view binding here
        binding.clickButton.setOnClickListener {
            click(100, 200)
        }
    }

    private fun click(x: Int, y: Int) {
        val gesture = GestureDescription.Builder().addStroke(
            GestureDescription.StrokeDescription(
                Path().apply { moveTo(x.toFloat(), y.toFloat()) },
                0, // startTime
                1, // duration
            )
        )
        dispatchGesture(gesture.build(), null, null)
    }
}

AndroidManifest.xml

<service
    android:name=".ui.ClickService"
    android:exported="false"
    android:permission="android.permission.BIND_ACCESSIBILITY_SERVICE"
    android:canPerformGestures="true">
    <intent-filter>
        <action android:name="android.accessibilityservice.AccessibilityService" />
    </intent-filter>
</service>

ClickService is then started as a foreground service from my MainActivity.

Running this (and giving the app its accessibility permissions) makes dispatchGesture return true, but it never simulates its click. Why is that? What's making this silently fail?

My thoughts on why this didn't work were:

  1. Perhaps there were additional permissions required that I was missing.
  2. Maybe dispatchGesture was running outside of the UI thread.

However:

  1. I already have canPerformGestures set to true in the service's manifest. Other than enabling accessibility for the app (which already have), I don't see any other required permissions.
  2. I've looked into this, and it seems like setOnClickListener's callback is already run in the UI thread, so I shouldn't need to do anything special to make dispatchGesture run from the same thread.

Why does dispatchGesture return true, but then doesn't actually click on anything?


Solution

  • The answer turned out to be simple. For me, I wasn't defining the AndroidManifest properly.
    android:canPerformGestures was set to true, but not where android reads it from.

    Here is the updated service in AndroidManifest.xml (notice the meta-data element):

    <service
        android:name=".ui.ClickService"
        android:exported="false"
        android:permission="android.permission.BIND_ACCESSIBILITY_SERVICE">
        <intent-filter>
            <action android:name="android.accessibilityservice.AccessibilityService" />
        </intent-filter>
        <meta-data
            android:name="android.accessibilityservice"
            android:resource="@xml/config_accessibility_service" />
    </service>
    

    Create a file in res/xml/config_accessibility_service.xml (or whatever you want to name it, but make sure it matches in your AndroidManifest:

    <accessibility-service xmlns:android="http://schemas.android.com/apk/res/android"
        android:canPerformGestures="true" />