I have a mouse position on the screen, for example (10, 10) and I want to translate that posistion to a control. How can I do that?
Example:
from pywinauto.application import Application
app = Application((backend="uia").start("notepad.exe")
dlg = app.top_window()
hardcoded_file_button_rec = dlg.File.rectangle() #<RECT L10, T10, R40, B40>
given_mouse_position = (10, 10)
found_file_button = search_by_position(app, given_mouse_position)
assert hardcoded_file_button_rec == found_file_button.rectangle()
Does pywinauto already has built-in function for that? On this question I found how to iterate over all controls in a window using pywinauto. So iterating over it, I can check if controls[i].rectancle().top == 10 and controls[i].rectancle().left == 10.
What is the right thing to do?
Method .from_point(x, y) is under development.
Workaround is here: issue #413.
As well as on StackOverflow: How to pass POINT structure to ElementFromPoint method in Python?
Clean code sample for any element:
from ctypes.wintypes import tagPOINT
import pywinauto
elem = pywinauto.uia_defines.IUIA().iuia.ElementFromPoint(tagPOINT(x, y))
element = pywinauto.uia_element_info.UIAElementInfo(elem)
wrapper = pywinauto.controls.uiawrapper.UIAWrapper(element)
wrapper is what you need probably since it's actionable object having methods like .invoke() etc.