Detecting Omega-like forehead wrinkles using opencv

Question

I'm working on a project to detect Omega-like wrinkles in the forehead (a diagnostic feature of depression). I found Hessian filter to be a good way to detect wrinkles based on this paper:

Yap, M. H., Alarifi, J., Ng, C., Batool, N., & Walker, K. (2019). Automated Facial Wrinkles Annotator. In Lecture notes in computer science (pp. 676–680). https://doi.org/10.1007/978-3-030-11018-5_5

I used skimage's hessian filter although not optimal according to this but it's enough in my case.

I applied the following in order:

1. Gaussian blur.
2. Hessian filter.
3. Closing operation.

Here's my code:

image = io.imread("1.jpg", as_gray=True)
image = cv2.GaussianBlur(image, (5,5), 0)
hessian_image = filters.hessian(image)

kernel = np.ones((5,5), np.uint8)
closing = cv2.morphologyEx(hessian_image, cv2.MORPH_CLOSE, kernel)

closing = np.array(np.divide(closing, np.amax(closing)), dtype=np.float64)
closing *= 255
closing = np.uint8(closing)

cv2.imshow("Closing", closing)
cv2.waitKey(0)

Here's an input image:

Here's the output image:

I cannot detect the Omega-like (or rectangle-like) shape using template matching since they tend to vary from one image to another. Do you have any other ideas ?

Answer 1

May be you can try to use connected components to analyze structures of each pattern. Something like this:

from skimage import measure, filters
import numpy as np

# segments by connected components
segmentation = measure.label(closing)

# finds areas of each connected component, omits background
areas = np.bincount(segmentation.ravel())[1:]
# finds threhsold to remove small components
t = filters.threshold_otsu(areas)
# finds connected component indexes, which areas greater than the threshold
indexes = np.where(areas > t)[0] + 1
# filters out small components
dominant_patterns = np.isin(segmentation, indexes) 

# this is applicable if the image well centered
# forehead center is approximately positioned at image center
# flip image by columns
dominant_patterns_flipped = np.flip(dominant_patterns, axis=1)
# finds intersection with flipped image
omega_like = dominant_patterns * dominant_patterns_flipped

Note: this is applicable if the image is well centered, i.e. forehead is centered in the image, and assumes the existence of vertical symmetry.

This will give you following image:

Now we can profile the image by rows and columns to calculate pixel occurrence per each row and column, by using following function:

import numpy as np

def row_col_profiles(image):
    """
    Returns pixels occurances per row and column
    """
    rows, cols = np.indices((image.shape))
    
    row_lengths = np.bincount(rows.ravel())
    number_of_pixels_on_each_row = np.bincount(rows.ravel(), image.ravel())
    row_profile = number_of_pixels_on_each_row / row_lengths

    col_lengths = np.bincount(cols.ravel())
    number_of_pixels_on_each_col = np.bincount(cols.ravel(), image.ravel())
    col_profile = number_of_pixels_on_each_col / col_lengths

    return row_profile, col_profile

row_profile, col_profile = row_col_profiles(omega_like)

You can plot that profiles like this:

import matplotlib.pyplot as plt

fig, axes = plt.subplots(1,2, figsize=(10, 5))
axes[0].plot(row_profile)
axes[0].set_title("Horizontal Line Profile")
axes[0].set_xlabel("Row Index")
axes[0].set_ylabel("Bright Pixel Occurance")
axes[0].grid()
axes[1].plot(col_profile)
axes[1].set_title("Vertical Line Profile")
axes[1].set_xlabel("Column Index")
axes[1].set_ylabel("Bright Pixel Occurance")
axes[1].grid()

You will get something like this:

To check whether we have omega-like pattern we can take some thresholds from that profiles, for example 0.2, and also to check at least we have 2 peaks at relatively same level in vertical profile (I've used -10% of maximum).

is_omega_like = row_profile.max()>=0.2 and col_profile.max()>=0.2 and len(np.where(col_profile>col_profile.max()*0.9)[0])>=2

You can also try to measure some properties and find some reasonable threshold on connected components. Please check out the documentation.