Search code examples
pythonnumpyopencvpython-mss

What is the color space produced by MSS sct.grab()?

Python 3.10, opencv-python==4.5.4.60

I'm having a hard time understanding the color format of screenshots taken by MSS.

Here is the portion of my screen that I am screenshotting (with correct/expected colors, taken manually):

enter image description here

Here is the screenshot code I'm working with:

with mss.mss() as sct:
    monitor = rect
    res = np.array(sct.grab(monitor))[:, :, :3] # remove alpha

When I use cv2.imsave() or cv2.imshow(), the screenshot looks as expected. If I attempt to create masks on this image based on [R, G, B] colors, it produces opposite results, implying that the default color space of MSS Screenshots is BGRA.

If I do the following:

res = cv2.cvtColor(res, cv2.COLOR_BGRA2RGB)
cv2.imshow("res", res)
cv2.waitKey(0)

It produces this:

enter image description here

However, then I am able to perform RGB masking properly.

I may have just answered my own question, but is it correct to say that MSS Screenshots are in BGRA format, and OpenCV expects BGR format - which would explain why RGB images have reversed colors when using cv2.imshow() and cv2.imsave()?

Solution

  • You are right, MSS uses BGRA:

    You can even find some tips for the conversion: https://python-mss.readthedocs.io/examples.html#bgra-to-rgb

    Indeed, I would be glad to enhance the documentation if you find something better :)