Interact with pc in python (mouse+screenshot)

Question

I'm writing a bot that interact with the pc. Briefly what I do is:

-Take a screenshot of the screen -recognize an object (with cv2 matchTemplate) on this screenshot
-make some mouse action using the position found (example: move the mouse pointer on an icon found in the previous point)
-restart from the first point

Now, I have to interact with some object in movement on the screen, and so I need a fast way to take screenshot, or an alternative way to do this job, what can I do?

Answer 1

You could use the module PyAutoGUI!

It has screenshot functions:

pyautogui.screenshot('my_screenshot.png')

Mouse movement and clicking:

pyautogui.moveTo(100, 200)
pyautogui.click()

And basic image location on screen:

button_location = pyautogui.locateOnScreen('button.png')

So if you wanted to click on a certain image on the screen you could run:

x, y = pyautogui.locateCenterOnScreen('form_button.png') # Gets coords of center of image
pyautogui.click(x, y)

As stated on their site:

On a 1920 x 1080 screen, the locate function calls take about 1 or 2 seconds. This may be too slow for action video games, but works for most purposes and applications.

So bear this in mind.