Search code examples
pythoncomputer-visionpyautoguipyscreeze

Is it possible to pyscreeze.locate(needleImage, haystackImage): without reading haystackImage from a file each time?


I'm currently using PyAutoGUI for the locate function that searches a needleImage on a haystackImage. The example that the documentation provides takes in a path to the image. However, I have a function that compares a series of needleImage to a single haystackImages, and reading the same image file over the number of times it needs to check is quite inefficient.

Is there a way to avoid reading the heystackImage each time? If not, are there any alternative to pyautogui/pyscreeze's locate function that uses bufferedImage?

...
checks = {
        "recieve.png": 'recieve',
        "next.png": 'next',
        "start.png": 'start',
        "black.png": 'loading',
        "loading.png": 'loading',
        "gear.png": 'home',
        "factory.png": 'factory',
        "bathtub.png": 'bathtub',
        "refit.png": 'refit',
        "supply.png": 'supply',
        "dock.png": 'dock',

        # SPE
        "spepage.png": 'spe',
        "expeditionpage.png": 'expedition',
        "sortiepage.png": 'sortie',
        "practice.png": 'practice',
        "practiceinfo.png": 'practice',

        "oquest.png": 'quest',
        "quest.png": 'quest'
    }
    for key in checks:
        if (detect(key, cache=True)):
            return checks[key]
def detect(imgDir, confidence=0.85, cache=False):
    if (pyautogui.locate(os.path.join('images', imgDir), 'images\\capture.jpeg', confidence=confidence)) is not None:
        return True
    else:
        return False

Solution

  • pyautogui.locate() also accepts numpy arrays and PIL images as inputs. You can read your haystack image into a numpy array (BGR) or PIL image, and pass that instead of the file name.

    def detect(imgDir, haystackImage, confidence=0.85, cache=False):
        if (pyautogui.locate(os.path.join('images', imgDir), haystackImage, confidence=confidence)) is not None:
            return True
        else:
            return False
    
    from matplotlib import image
    hsImage = image.imread('images\\capture.jpeg')
    hsImage = hsImage[:,:,::-1] # convert RGB to BGR
    detect('needleImg.png', hsImage, cache=True)
    
    # Alternate method
    from PIL import Image
    hsImage = Image.open('images\\capture.jpeg')
    detect('needleImg.png', hsImage, cache=True)
    
    

    The second method can be slower than the first since pyautogui.locate() ultimately loads the PIL image as a numpy array, which requires additional processing.