Search code examples
androidocrtasker

Detect x y coordinates of specific text

I tried write an automation for my games in Android with Tasker and AutoTools plugins. Somethings ok at this point but i need capture the screenshot and need interpret it for my needs.

That's exactly what I need;

Some texts are important in games and i want to click on it wherever they are in the screen. So i need OCR for this task i think. I follow some solutions but fail or stuck everytime. Let me explain which solutions i tried.

Following Solution 1:

  • I tried AutoInput (Tasker plugin) UIQuery method but fail. Because UIQuery of AutoInput just works on android UI i think. Cant get any information from 3D App like games.

Following Solution 2:

  • I search OCR solution and find AutoTools (Tasker plugin)

  • Create a task and take screenshot and interpret it with AutoTools OCR method. Thats ok. AutoTools OCR succesfully read a text from Image file.

  • But i stuck again. Because i succesfully read a text from image file but i dont know x y coordinate of important text.

What suggest at this point?

Should i learn android and write own app?

Solution

  • I contact with the developer who write the "AutoTools" Tasker plugin.

    He/She add some function to plugin and solve it.

    Plugin, interpret with OCR granted image and return words and center of xy positions of each words now.

    If anyone search like this function for Android and Tasker App please visit this forum topic link. Its very useful.