Retrieve TYPE_TOUCH and TYPE_GESTURE in AccessibilityService

Question

I am developing an application which is supposed to collect user's interaction with Android device (touch related data) similar to this SO post.

I've tried all of the answer's suggested approaches starting from an app overlay with and without WATCH_OUTSIDE_TOUCH flag. getevent approach is also not an option for me as it requires root or external storage (as the answer states).

So I've created an AccessibilityService and now I am able to detect such events as: VIEW_CLICKED, VIEW_SCROLLED. However in the documentation I've encountered also GESTURE_DETECTION_START, GESTURE_DETECTION_END, TOUCH_INTERACTION_STARTand TOUCH_INTERACTION_END. I would like to use them to measure the interaction time. Unfortunately my service is not retrieving this events at all despite proper (?) configuration. My current configuration looks like:

AndroidManifest.xml:

...
<service
    android:name=".accessibility.MyAccessibilityService"
    android:label="@string/accessibility_service_label"
    android:permission="android.permission.BIND_ACCESSIBILITY_SERVICE">
    <intent-filter>
        <action android:name="android.accessibilityservice.AccessibilityService" />
    </intent-filter>
    <meta-data
        android:name="android.accessibilityervice"
        android:resource="@xml/accessibility_config" />
</service>
...

accessibility_config.xml:

<?xml version="1.0" encoding="utf-8"?>
<accessibility-service xmlns:android="http://schemas.android.com/apk/res/android"
    android:accessibilityEventTypes="typeAllMask"
    android:accessibilityFeedbackType="feedbackAllMask"
    android:description="@string/accessibility_service_description"
    android:notificationTimeout="5"
    android:summary="@string/accessibility_service_description" />

Summing up, my service is starting and retrieving some events properly, but it lacks gesture and touch start/end events. I've tried many other configurations including all capabilities and flags enabled but without any effect. Also I tried overriding a method AccessibilityService::onGesture but it's never called. Are this events available only in touch exploration mode?

I've described my problem maybe a little too broad, but I would appreciate any help or suggestions even not related strictly to AccessibilityService approach.

Answer 1

After some struggling I found out what was the problem (or set of problems):

• My accessibility_config.xml file was invalid because it contained <?xml version="1.0" encoding="utf-8"?> tag, and according to the docs

This meta-data must reference an XML resource containing < accessibility-service > tag

So I deducted that it needs this and only this tag to work correctly.

• GESTURE_DETECTION_START, GESTURE_DETECTION_END, TOUCH_INTERACTION_START and TOUCH_INTERACTION_END are events available only in touch exploration mode.

So android:canRequestTouchExplorationMode="true" and android:accessibilityFlags="flagRequestTouchExplorationMode" were missing in my config. The docs enlist those events under Exploration types.

EDIT: sharing the final accessibility_settings.xml file:

<accessibility-service xmlns:android="http://schemas.android.com/apk/res/android"
    android:accessibilityEventTypes="typeViewTextChanged|typeViewFocused|typeViewLongClicked|typeViewClicked|typeViewScrolled|typeWindowStateChanged"
    android:accessibilityFeedbackType="feedbackHaptic"
    android:accessibilityFlags="flagIncludeNotImportantViews"
    android:canRetrieveWindowContent="true"
    android:description="@string/accessibility_service_description"
    android:notificationTimeout="10"
    android:settingsActivity="org.example.ExampleActivity" />