Search code examples
androidshelladb

How to translate device screen position to sendevent position?


I know about the input tap x y shell command, however, I'm trying to understand how to perform a click using the sendevent command. I been able to achieve it with the following command:

sendevent /dev/input/event5 3 53 X &&
sendevent /dev/input/event5 3 54 Y &&
sendevent /dev/input/event5 0 2 0 &&
sendevent /dev/input/event5 0 0 0 &&
sendevent /dev/input/event5 0 2 0 &&
sendevent /dev/input/event5 0 0 0

Where X and Y is the position that will be clicked, I'm testing it on the android emulator BlueStacks 5 which the Display Resolution set to 1920x1080.

The code is working and the click is fired, however, I couldn't understand how to convert the position where I want to be clicked to the sendevent XY position.

If I send using ADB:

sendevent /dev/input/event5 3 53 2000 &&
sendevent /dev/input/event5 3 54 2000 &&
sendevent /dev/input/event5 0 2 0 &&
sendevent /dev/input/event5 0 0 0 &&
sendevent /dev/input/event5 0 2 0 &&
sendevent /dev/input/event5 0 0 0

It clicks somewhere around x75 y75, how this calc is done? i mean screen xy -> sendevent xy?


How to replicate:

  • First enable BlueStacks 5 adb in the window: Settings -> Advanced -> Android debug bridge
  • Open a cmd window and run cd C:\Program Files\BlueStacks_nxt assuming BlueStacks where installed in the default path.

Execute the commands:

  • hd-adb.exe connect 127.0.0.1:X where X is the port shown in the window where you enabled the ADB.
  • hd-adb.exe -s 127.0.0.1:X shell

Now we are on the shell, execute a new command: getevent -p and search for:

... /dev/input/event5
  name:     "BlueStacks Virtual Touch"

On my emulator the input event for touch is event5 on yours it can be different, replace it according.

Now you can simulate a click with the code below changing XY to the position where you want to be clicked:

    sendevent /dev/input/event5 3 53 X &&
    sendevent /dev/input/event5 3 54 Y &&
    sendevent /dev/input/event5 0 2 0 &&
    sendevent /dev/input/event5 0 0 0 &&
    sendevent /dev/input/event5 0 2 0 &&
    sendevent /dev/input/event5 0 0 0

I'm trying to figure out how to convert the emulator screen position to the sendevent position.

For example, if you want to perform a click at x200 y200, using sendevent what the value needed?

How to calculate it?


Solution

  • thanks for the very precise instructions to reproduce:

    I enabled
    Settings -> Advanced -> Input debugging -> Show visual feedback for taps
    and
    Settings -> Advanced -> Input debugging -> Show pointer location for current touch data

    when I hold down click, I can see: X: Y:

    I collected these x coordinate points, my max X: is 1600.0 so my width is 1600.0

    (8000, 390.6)
    (16000, 781.2)
    (32000, 1562.5)
    

    then Excel: X Y (scatter) chart, add trendline, click trendline and click big + sign -> Chart Elements -> Trendline : (tick that) and ▶, More Options... -> Trendline Options -> Display Equation on chart
    , click on the formula, then Label Options -> Category: Number, Decimal Places: 10
    y = 0.0488294643x - 0.0500000000

    1600 = 0.0488294643x - 0.0500000000
    32768.1252075501 = X

    I round to 32768 because 32768 is a magic number, close to Int16's 32767
    edit: after Nathan's comment, it's 32767 (32768 doesn't even move the cursor)

    so the formula is : (W: width, H: height)

    32767*X/W
    32767*Y/H
    

    for your (X=200, Y=200, W=1980, H=1080)

    32767*200/1920
    32767*200/1080
    
    3413.22916666667
    6067.96296296296
    

    I used this code to test: it doesn't do a tap, it holds down without releasing

    sendevent /dev/input/event5 3 57 0
    sendevent /dev/input/event5 3 53 3413.22916666667
    sendevent /dev/input/event5 3 54 6067.96296296296
    sendevent /dev/input/event5 3 48 5
    sendevent /dev/input/event5 3 58 50
    sendevent /dev/input/event5 0 2 0
    sendevent /dev/input/event5 0 0 0
    

    https://ktnr74.blogspot.com/2013/06/emulating-touchscreen-interaction-with.html#:~:text=ABS_MT_TRACKING_ID%20(57)%20%2D%20ID,end%20of%20report