Search code examples
pythonautomationscreenpyautogui

Run a python code only when a specific object appears in screen

In the following image, I want Python (using module Pyautogui) to click the "OK" button.

The problem is, I dont know when the "OK" button appears. So how do I make python to click the "OK" button only when it appears?

References:

If you wonder it looks like a mobile screen, I am using an android emulator.

Solution

  • PyAutoGUI has a function called locateOnScreen() which takes a screenshot of your current screen and then looks for the provided image you pass into the parameter of the function.

    First, you need to have a picture of the "OK" button to tell PyAutoGUI what to look for, which you can do using pyautogui.screenshot('ok.png', region=(0, 0, 300, 400)) the region parameter specifies where to look and the size of the picture.

    After that, you can run locateOnScreen('ok.png') and if the picture 'ok.png' is currently on your monitor it will return the center coordinates, which you can then pass to pyautogui.click()

    import pyautogui

pyautogui.screenshot('ok.png', region=(0, 0, 300, 400))

    Run that line of code and play around with the region parameter until you get a good looking 'ok.png' and make sure only the button is visible in the picture as PyAutoGUI looks for an exact match. Once you're done, replace the line with the following:

    import pyautogui

location = pyautogui.locateOnScreen('ok.png')

pyautogui.click(location)

    Now, whenever the "OK" button is on your screen, PyAutoGUI will click on it