Search code examples
opencvscreenshotcomputer-visionvideo-capture

Detecting video playing in browser from a screenshot -- OpenCV

I would like to draw a rectangle around a video playing on my screen. For example, I am watching a YouTube video in my browser. I would like to be able to take a screenshot, analyze that screenshot, and then draw a rectangle around where the YouTube video is playing. I have just started looking into how I might be able to to this. I came across OpenCV. I understand that OpenCV covers many computer vision techniques. Would any of them be particularly well suited for this task? Also, is this something that can be done in real time? Finally, is there a technique that would work for both in browser and full screen?

Thanks!

Edit:

Marc makes a good point. Ideally I would like to be able to identify the video using just one screenshot. I am not sure if that if possible though. How about comparing screenshots a taken a second or two apart?

Solution

  • If the video is not playing in full-screen, you can:

    1. Detect moving parts (simple optical flows
    2. Perform template matching for the video controls :D (much simpler than YouTube logos, ...)
    3. Morphological operations on the flow fields.

    If the video is playing in full-screen, an SVM trained on the histogram of optical flow should tell you that it is full-screen.