Read text from user input (in other applications) using an Accessibility app (for visually impaired people)

Question

In an application that is meant to help visually impaired people, I declare the accessibility service as follows:

<manifest ...">

    <application
        android:name=".application.TxExApplication"
        ...
        android:theme="@style/Theme.AppCompat.Light"
        tools:targetApi="31">
        <activity
            android:name=".activity.MainActivity"
            android:exported="true"
            android:label="@string/app_name"
            android:theme="@style/Theme.AppCompat.Light">
            <intent-filter>
                <action android:name="android.intent.action.MAIN" />

                <category android:name="android.intent.category.LAUNCHER" />
            </intent-filter>
        </activity>
        <service android:name=".service.TextExpansionAccessibilityService"
            android:permission="android.permission.BIND_ACCESSIBILITY_SERVICE"
            android:label="@string/accessibility_service_label"
            android:enabled="true"
            android:exported="true">
            <intent-filter>
                <action android:name="android.accessibilityservice.AccessibilityService" />
            </intent-filter>
            <meta-data
                android:name="android.accessibilityservice"
                android:resource="@xml/accessibility_service_config" />
        </service>
    </application>
    <uses-permission android:name="android.permission.INTERNET" />
</manifest>

My AccessibilityService attempts to read the last word of the text field where the current cursor is placed (I haven't tested with webview applications but at the moment my focus is on native applications).

public class TxExAccessibilityService extends AccessibilityService {

    @Override
    protected void onServiceConnected() {
        AccessibilityServiceInfo info = new AccessibilityServiceInfo();
        info.eventTypes = AccessibilityEvent.TYPES_ALL_MASK;
        info.feedbackType = AccessibilityServiceInfo.FEEDBACK_ALL_MASK;
        info.notificationTimeout = 100;
        info.packageNames = null;
        setServiceInfo(info);
    }

    @Override
    public void onAccessibilityEvent(AccessibilityEvent event) {

        if (event.getEventType() == AccessibilityEvent.TYPE_VIEW_TEXT_CHANGED) {
            AccessibilityNodeInfo source = event.getSource();
            if (source == null) {
                return;
            }

            List<CharSequence> textList = event.getText();
            for (CharSequence text : textList) {
                if (text != null && text.length() > 0) {
                    Log.d("AccessibilityService", "Retrieved text: " + text);
                    String [] words = text.toString().split("\\s+");
                    String last = words[words.length-1];

                    ...
                }
            }
        }
    }

    @Override
    public void onInterrupt() {

    }

}

I have enabled accessibility for my app in settings but unlike accessibility applications I have worked with in the past, my app is not reading text from user input. Does anyone know how to read text from user input in modern versions of the Android operating system?

Answer 1

I think there is nothing wrong with the code you shared in the question because it works well on my side, with the code lab.

I can get the following logcat when I'm inputting an EditText's text:

D/AccessibilityService: Retrieved text: asfsdafasdfasdfsdfg

So I will give some tips that may be missed:

• Don't forget to add the android:canRetrieveWindowContent="true" to your accessibility_service_config, like:

  <accessibility-service 
  xmlns:android="http://schemas.android.com/apk/res/android"
      android:accessibilityFeedbackType="feedbackGeneric"
      android:accessibilityFlags="flagDefault"
      android:canPerformGestures="true"
      android:canRetrieveWindowContent="true" />

• Every time you re-run your app, you have to re-open the device's accessibility setting for your app.

• Also it is suggested to add some debug logs to your key methods for debuging, like:

    @Override
    protected void onServiceConnected( ) {
        Log.i("AccessibilityService", "onServiceConnected");

    @Override
    public void onAccessibilityEvent(AccessibilityEvent event) {
        Log.i("AccessibilityService", "onAccessibilityEvent: " + event.toString());