Get capture from fullscreen

Question

I'm trying to make a "video" of my screen, so that the opencv can analyze it, and so far it works well, however when I go into full screen mode, the program stops recording or analyzing, this stops working. Can you help me?

from mss import mss
import cv2
from PIL import Image
import numpy as np
from time import time
from pynput.mouse import Button, Controller

mon = {'top': 370, 'left':675, 'width':15, 'height':20}
mouse = Controller();
sct = mss()
upper_purple = np.array([[255, 81, 255]])
lower_purple = np.array([[96, 35, 84]])
while 1:
    
    
    begin_time = time()
    sct_img = sct.grab(mon)
    img = Image.frombytes('RGB', (sct_img.size.width, sct_img.size.height), sct_img.rgb)
    img_hsv = cv2.cvtColor(np.array(img), cv2.COLOR_BGR2RGB)
    height, widht, channel_color = img_hsv.shape
    cv2.imshow('test', np.array(img_hsv))
    for y in range (0, height):
        for x in range (0, widht):
            red = img_hsv.item(y, x, 0)
            green = img_hsv.item(y, x, 1)
            blue = img_hsv.item(y, x, 2)
            offset = red - blue
            if ((img_hsv[y][x] >= lower_purple).all() and (img_hsv[y][x] <= upper_purple).all()):
                mouse.click(Button.left, 1)
                print("SHOOT")
    print('This frame takes {} seconds.'.format(time()-begin_time))
    
    if cv2.waitKey(25) & 0xFF == ord('q'):
        cv2.destroyAllWindows()
        break

Answer 1

Pythons mss package seems to be unable to handle gpu output in some cases. Depending on the game and the implementation the fullscreen mode cant be handled and mss only capture "windowed" mode. I had the same issue just captuing minecraft on Win10 running NVIDIA GeForce RTX 3070 Ti.
Found some module called dxcam which seems to work well and recognize the gpu output.
I tried this code in python 3.11 and it worked. The documentation is a little bit unclear, but it works.

import cv2
import dxcam

print(dxcam.device_info())
camera = dxcam.create()

# Start capturing frames
camera.start()

while True:
    # Get the latest captured frame from the camera
    frame = camera.get_latest_frame()

    if frame is not None:
        # Convert the BGR frame to RGB
        corrected_frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)

        # Display the corrected frame using OpenCV
        cv2.imshow('Corrected Frame', corrected_frame)

        # Exit the loop and close the window when 'q' is pressed
        if cv2.waitKey(1) & 0xFF == ord('q'):
            break

# Stop capturing when done
camera.stop()

# Close OpenCV windows
cv2.destroyAllWindows()

I hope this helps, as the question is not that clear.