android talkback accessibility - customizing sounds

Question

I've created my own set of alphabetical sounds (replacing the regular a,b,c,..z). I'd like to customize the talkback functionality so it uses my own sounds instead of the built-in ones when using my own app. This means that when a visually impaired user would try to read the content of a label in my app he'd hear my own sounds. Is this possible? If so, what would be the right way to achieve this?

Answer 1

Assuming your own sounds have a "phonetic spelling" this would be pretty easy. Let's say you had the letter A and you wanted it to pronounce as "A" and not the sound "uh". You could simply replace "A" with "ay" and TalkBack would pronounce it correctly. Assuming this is the case, what you want to do is very easy. If what you have created is actual sounds, and cannot simply use phonetic spellings as I am assuming, as alanv has said, this is impossible. Or at least, involves more than just changes to your app!!!

What you want to do is intercept all accessibility events coming from your application, and then when you intercept the events, replace the content description with your phonetically spelled content description. The tricky part is emulating TalkBack logic for grabbing text from Accessibility events, so that you grab the correct text! Otherwise you end up modifying the wrong string, or just nothing.

If you attach this accessibility delegate to the views within your view heirarchy, you can override the content description of the accessibility node infos, and replace it with your phonetic pronunciations. I attached all relevant portions of my solution. There may be a way to get this working by only mucking with the root view's accessibility delegate, and not the entire view hierarchy. I may investigate more later, but this works just dandy, and is a linear operation on load (and view addition for dynamic content) which isn't bad at all.

Add this code to your onCreate method, and modify the "convertText" function to suit your needs, and you should be all set!

final View.AccessibilityDelegate accessiblityDelegate =  new View.AccessibilityDelegate() {

    String convertText(String argString) {
        //Do your phonetic conversion in here!
        //A little Regex.  A little String replacement and you're golden!
        return argString;
    }
    @Override
    public void onInitializeAccessibilityNodeInfo(View host, AccessibilityNodeInfo nodeInfo) {
        super.onInitializeAccessibilityNodeInfo(host, nodeInfo);

        String text = null;

        if (nodeInfo.getContentDescription() != null) {
            text = convertText(nodeInfo.getContentDescription().toString());
        } else if (nodeInfo.getText() != null) {
            text = convertText(nodeInfo.getText().toString());
        } else if (host instanceof TextView) {
            TextView textView = (TextView)host;
            text = convertText(textView.getText().toString());
        }

        if (text != null) nodeInfo.setContentDescription(text);
    }

};

rootView.addOnLayoutChangeListener(new View.OnLayoutChangeListener() {
    @Override
    public void onLayoutChange(View v, int left, int top, int right, int bottom, int oldLeft, int oldTop, int oldRight, int oldBottom) {
        addAccessibilityDelegateToViews(v);
    }

    private void addAccessibilityDelegateToViews(View v) {
        v.setAccessibilityDelegate(accessiblityDelegate);

        if (v instanceof ViewGroup) {
            ViewGroup viewGroup = (ViewGroup)v;

            for (int i = 0; i < viewGroup.getChildCount(); ++i) {
                View view = viewGroup.getChildAt(i);

                addAccessibilityDelegateToViews(view);
            }
        }
    }
});