Ideas for Extracting Blade Tip Coordinates from masked Wind Turbine Image

Question

I am looking for image processing tools in python to get the coordinates from the Blade Tips of a Wind Turbine, in this case a small model of one. The blades already get segmented by a yoloV8 Segmantation Model, and now I want to use that image to get the xy coordinates of the tips. Example image: masked wings of a wind energy turbine. Can someone recommend some ideas how I could go about this? The rotor can be rotated, so the three tips could be anywhere on an ellipse.

I already tried training a yolo-pose model for keypoint detection, but it didn't give precise enough results. I will use these coordinates to calculate the excentricity of the rotor disk, so the points need to be somewhat precise.

Answer 1

You'll need several processing to robustly find the tips coordinates of the turbine, no matter its rotation. Basically you want to find the edges of the turbine, then compute the center of the shape and cluster each points of the contour in 3 groups (cause there is 3 blades). You can do everything using opencv and numpy

Loading the image

We use opencv to load the image and rasterize the colors to pure black or white. This will later help us find the contour:

def load_image(path):

    # Load the image
    image = cv2.imread(path)
    # Convert to grayscale
    gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
    # Threshold the image to get a binary image
    _, binary = cv2.threshold(gray, 1, 255, cv2.THRESH_BINARY)

    return image, binary

It's a good start but as you can see the mask you got from yolo isn't 100% perfect. We'll easily fix that while computing the contours.

Compute the turbine contour

We can use cv2.findContours function to compute the different contours of the binary image. This will returns the contour of the turbine as well as the mask artefacts. So we will only keep the contour with the largest area, assuming that artifacts are smaller than the actual turbine:

def find_contour(binary):

    # Find the contours
    contours, _ = cv2.findContours(binary, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)

    # Since the masking is not perfect, we have several contours
    # Assuming the largest contour area corresponds to the turbine
    if len(contours) > 0:
        largest_contour = max(contours, key=cv2.contourArea)
        return largest_contour
    
    # If no contours, return False
    return False

Find the tips coordinates

In order to find the tips we'll cluster the contour points on 3 groups, for the 3 blades. So let's first find the center of the turbine:

def find_tips(contour):

    # Calculate the centroid of the contour
    M = cv2.moments(contour)
    cx = int(M['m10'] / M['m00'])
    cy = int(M['m01'] / M['m00'])
    centroid = (cx, cy)

    [...]

Then we need to compute the angles and distances from the center for each points. I also sort the angles_distances list by angle:

def find_tips(contour):

    [...]

    # Calculate angles and distances from centroid
    angles_distances = []
    for point in contour:
        x, y = point[0]
        angle = angle_from_centroid(centroid, (x, y))
        distance = np.sqrt((x - cx)**2 + (y - cy)**2)
        angles_distances.append((angle, distance, (x, y)))

        # Sort by angle
        angles_distances.sort()

    [...]

Here is my angle_from_centroid function to calculate the angle (in radians):

def angle_from_centroid(centroid, point):
    # Compute the angle of the vector from centroid to the point
    angle = np.arctan2(point[1] - centroid[1], point[0] - centroid[0])
    return angle

We can finally split our points into 3 cluster and save the farthest point of each cluster, these are our tips coordinates:

    # Divide into three clusters and find farthest point in each
    num_points = len(angles_distances)
    tips = []
    for i in range(3):
        cluster = angles_distances[i * num_points // 3: (i + 1) * num_points // 3]
        farthest_point = max(cluster, key=lambda x: x[1])[2]
        tips.append(farthest_point)
            
    return tips

This last part is a bit complexe, I'm not getting to deep into it. But we can now draw the tips of the blades.

Final result

We can now run the 3 functions described above and draw the tips coordinates on the image:

image, binary = load_image('resources/trBo33Jy.png')
contour = find_contour(binary)
coordinates = find_tips(contour)

# Draw the tips coordinates on the image
for point in coordinates:
    cv2.circle(image, point, 20, (0, 255, 0), -1)
    print(point)

# Display the image with marked points
cv2.imshow('Image with Tips', image)
cv2.waitKey(0)
cv2.destroyAllWindows()

If you're looking for the most precise result possible, you could try to calculate the average angle of each cluster to get the vector of the blades