~ Identical question here, but sadly no useful answers.
I've used PlayStation Remote Play to stream live video from my PS5 to my computer. The stream is being shown in a window called 'PS Remote Play' on my computer. I would like to read frames from this window for eventual processing in opencv.
Using libraries mss and pygetwindow, I am able to capture this window correctly:
import cv2
import mss
import pygetwindow as gw
import numpy as np
def fetch_window_image(window_name):
#
window = gw.getWindowsWithTitle(window_name)[0]
window_rect = {'left': window.left + 10, 'top': window.top + 45,
'width': window.width - 20, 'height': window.height - 60}
with mss.mss() as sct:
screenshot = sct.grab(window_rect)
img = np.array(screenshot)
img = cv2.cvtColor(img, cv2.COLOR_BGRA2BGR)
return img
def main():
WINDOW_NAME = 'PS Remote Play'
cv2.namedWindow('Computer Vision', cv2.WINDOW_NORMAL)
while cv2.waitKey(1) != ord('q'):
img = fetch_window_image(WINDOW_NAME)
cv2.imshow('Computer Vision', img)
if __name__ == '__main__':
main()
The problem is that this method is unable to capture it when the window is obscured by other windows (while remaining active). Being able to capture hidden windows is essential for my use case.
I came across the following code, which is able to capture hidden windows (source):
import win32gui
import win32ui
import win32con
w = 1920
h = 1080
hwnd = win32gui.FindWindow(None, 'PS Remote Play')
wDC = win32gui.GetWindowDC(hwnd)
dcObj = win32ui.CreateDCFromHandle(wDC)
cDC = dcObj.CreateCompatibleDC()
dataBitMap = win32ui.CreateBitmap()
dataBitMap.CreateCompatibleBitmap(dcObj, w, h)
cDC.SelectObject(dataBitMap)
cDC.BitBlt((0, 0), (w, h), dcObj, (0, 0), win32con.SRCCOPY)
dataBitMap.SaveBitmapFile(cDC, 'saved.bmp')
# Free Resources
dcObj.DeleteDC()
cDC.DeleteDC()
win32gui.ReleaseDC(hwnd, wDC)
win32gui.DeleteObject(dataBitMap.GetHandle())
Unfortunately, when used on the PS Remote Play window, it shows a black screen. I tried it on some other windows, and results were mixed. It was able to capture Spotify, Paint and the VSCode editor. It returned blank screens for PS Remote Play, Microsoft To Do, Whiteboard and Calendar. These behaviours remain the same whether visible or not.
Why is this happening, and how can we consistently capture hidden windows?
Quite a lot of games have multiple window modes, some of which blank the screen when not in focus to save on graphics a little. I am not sure if this setting exists in PS Remote Play, but try looking into the game settings and find an option that says "window mode" or "display mode" and select "windowed fullscreen" or "borderless" if you see it.
I encountered a similar issue quite a while ago with OBS not broadcasting a specific out-of-focus window correctly and this was how I solved it. Since your code works with other hidden windows, this is probably your issue too.