OpenCV contour detection: Extract rectangular video element out of entire Youtube webpage

Question

Working on extracting the rectangle hosting the Youtube video element out of the entire screenshot of webpage. I'm struggling to find any solution that works with good accuracy across different videos and page form factors.

Attaching a sample instance I am interested in-

Here's the most promising solution I've found yet:

1. Thresholding

2. FindContours

3. Look in Largest Bounding Rects

    gray = cv2.cvtColor(src, cv2.COLOR_BGR2GRAY)
    gray = clahe(gray, 5, (3, 3))
    img_blur = cv2.blur(gray, (9, 9))
    img_th = cv2.adaptiveThreshold(img_blur, 255, cv2.ADAPTIVE_THRESH_GAUSSIAN_C, cv2.THRESH_BINARY, 51, 2)
    contours, hierarchy = cv2.findContours(img_th,cv2.RETR_CCOMP,cv2.CHAIN_APPROX_SIMPLE)
   

Contours Look like:

Large contours(boundingRect) found:

What I wanted out of the image was the central large blue rectangle hosting video content. Clearly not showing up in detected contours by default.

Tried Canny as well instead of thresholding but no good. Any suggestions on what approach I can take to solve this problem? (on how to handle detected contours or maybe generate a more filtered set of contours)

I feel this problem should be simpler than many image processing task involving real world images as objects here have perfect boundaries.

Answer 1

Able to find a solution to this use case, Thanks to @fmw42. Sharing the solution here and further insights for solving similar tasks.

1. Binary Thresholding:

InRange- since we are only interested in getting the rectangle and not its inner details. This reduces the problem better than adaptive thresholding.

2. Morphology:

Open MORPH_OPEN (in this case) to cleanup noise.

3. Contour Detection:

gray = cv2.inRange(gray, 0, 20)
showImageWithResize('InRange', gray)
gray = cv2.bitwise_not(gray)
showImageWithResize('BinaryImage', gray)


kernel = np.ones((5, 5), np.uint8)
gray = cv2.morphologyEx(gray, cv2.MORPH_OPEN, kernel, iterations=2)

showImageWithResize('Morphology', gray)

contours, _ = cv2.findContours(gray, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)

result_image = cv2.drawContours(image.copy(), contours, -1, (0, 255, 0), 5)
showImageWithResize('contours', result_image)

contours = sorted(contours, key=cv2.contourArea, reverse=True)
showImageWithRects("rect", image.copy(), [cv2.boundingRect(contours[0])])

cv2.waitKey(0)
cv2.destroyAllWindows()