Intercept all touch data (including keyboard)

Question

I am currently investigating the ways in which touchscreen data can be retrieved in android.

Before I continue, the app will be used for research purposes and will be utilising all sensory inputs for an experiment. The app is not intended for public release. Users will know all of their inputs are being monitored.

I basically want to monitor all touch inputs on the phone screen, including keyboard presses etc. I have been able to get general touch events:

public override bool OnTouchEvent(MotionEvent e)
{
    int x = (int)e.GetX();
    int y = (int)e.GetY();

    System.Diagnostics.Debug.WriteLine($"Touch detected at x:{x} y:{y}");

    return base.OnTouchEvent(e);
}

But this method won't get called for the following situations:

1. Another item in the view is tapped (not the view itself, ie a button).
2. The keyboard is presented

I want to be able to get touch data for any touch that occurs on the phone's screen. I have included a screenshot below of the sort of screen tap's I am interested in capturing.

Is this possible with the default keyboard? I'm likely to use Xamarin Android to implement this project (as I've alot more experience of xamarin than I do of android). I'd much prefer to stay as vanilla as possible.

Thanks

Answer 1

I have been able to 'bodge' together a solution that works perfectly for my needs. The solution isn't perfect and requires the android device to be connected to a computer and a series of scripts need to be run.

Android Touch Record Replay provided exactly all the steps needed to extract the device touch data into a csv file.

To get touch event data you must do the following:

• Run the shell script ./find_touchscreen_name.sh If this works ignore the next step
• Run the command adb shell getevent -lp (you may need to replace shellfor exec-out
• Determine which device is your touchscreen by running adb shell getevent | grep event<your_index> for each index that contains the ABS_MT_TOUCH_MAJOR label and monitoring the output.
• Once you have the touch screen call the python script: human_readable_data.py with the following arguments:

Usage: python human_readable_data.py arg1 arg2

| Argument | Value               | Example                             |
| -------- | ------------------- | ----------------------------------- |
| arg1     | touchscreen device  | /dev/input/event7                   |
| arg2     | csv output location | /Users/$user/Desktop/touch_info.csv |

• Record your touch screen taps
• Close the python script (ctrl + c)
• Open the output csv

Now we have recorded our android touch screen data for further analysis.

This solution does NOT required a rooted device!