Detect known fixed shape on the image with different backgrounds

Question

I am trying to determine if an image contains a specific shape of a hourglass. The shape is always in the fixed location and has a fixed size. However, the background is different and it affects the color of the shape. See examples 1 , 2 and 3: , ,

If the hourglass is empty, the image contains only the background. What is the best way to detect such shape minimizing false-positives and false-negatives?

The thresholding (both static and adaptive) is not very reliable it seems since the following code doesn't work as good for the second example:

Image with some parts missing:

import cv2

img = cv2.imread('hourglass.png')
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
thresh = cv2.threshold(gray, 70, 255, cv2.THRESH_BINARY)[1]
cv2.imshow("thresh", thresh)
cv2.waitKey(0)

I thought about manually checking that the pixels where the hourglass should be are close in color, but it seems "hacky" and may give false results depending on the choice of threshold

Answer 1

I made a (rather crude) mask of the interesting pixels that constitute the hourglass in your image as follows - mask.png:

Then I converted your image to HSV colourspace and separated out the Saturation channel. In general, this will be high where an image is saturated and vivid, and low where an image is less vivid, less saturated and more grey. I then calculated the mean and standard deviation of the image in the masked area, and then once again in the area outside the masked area.

I am positing that the mean will be low where the hourglass is because that is unsaturated and that the variance will also be low where the hourglass is because there is little variation within the hourglass. Threshold the results to match your data.

#!/usr/bin/env python3

import cv2 as cv

# Load mask with interesting pixels masked as white
mask = cv.imread('mask.png', cv.IMREAD_GRAYSCALE)

def processOne(filename, mask):
    # Load image, convert to HSV, and split out S component
    im = cv.imread(filename)
    HSV = cv.cvtColor(im, cv.COLOR_BGR2HSV)
    _, S, _ = cv.split(HSV)
    # Calculate masked mean and stddev of S component
    mean, stddev =  cv.meanStdDev(S,mask=mask)
    print(f'File: {filename}, MASKED AREA {mean=}, {stddev=}')
    mean, stddev =  cv.meanStdDev(S,mask=~mask)
    print(f'File: {filename}, UNMASKED AREA {mean=}, {stddev=}')

processOne('h1.png', mask)
processOne('h2.png', mask)
processOne('h3.png', mask)

---

Output:

File: h1.png, MASKED AREA mean=array([[13.89722222]]), stddev=array([[10.40768909]])
File: h1.png, UNMASKED AREA mean=array([[74.00181488]]), stddev=array([[33.2070991]])
File: h2.png, MASKED AREA mean=array([[55.66944444]]), stddev=array([[22.63463815]])
File: h2.png, UNMASKED AREA mean=array([[144.43647913]]), stddev=array([[18.2676475]])
File: h3.png, MASKED AREA mean=array([[15.55833333]]), stddev=array([[12.96799211]])
File: h3.png, UNMASKED AREA mean=array([[96.01270417]]), stddev=array([[32.66029606]])