how to detect square in image with noisy background

Question

I have several image of seedling plots. I want to run a code to slice each square pot and store it.

To do this, first convert it to gray image then to reduce noise bluer it and finally convert it to binary using threshold.

Now I am trying to find geometry location of each square using cv.findContours but seems just can return points in corner of image instead finding corner of each square.

gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
blurred = cv2.GaussianBlur(gray, (3, 3), 0)
edges = cv2.Canny(blurred, 10, 50)

thresh = cv2.adaptiveThreshold(gray, 400, cv2.ADAPTIVE_THRESH_MEAN_C, cv2.THRESH_BINARY, 7, 4)
lines = cv2.HoughLines(thresh, rho=1, theta=np.pi/180, threshold=100)
contours, hierarchy = cv2.findContours(edges, cv2.RETR_LIST, cv2.CHAIN_APPROX_NONE) 
squares = []
for contour in contours:
    peri = cv2.arcLength(contour, True)
    approx = cv2.approxPolyDP(contour, 0.02 * peri, True)
    if len(approx) == 4:
        x, y, w, h = cv2.boundingRect(approx)
        aspect_ratio = float(w) / h
        if 0.98 <= aspect_ratio <= 1:  # Adjust this range as per your requirement
            squares.append(approx)
# Draw squares on the original image
for square in squares:
    cv2.drawContours(image, [square], -1, (0, 255, 0), 2)

The problem with result is that I get many point that is not exactly the border of each pot. I would appreciate for your tips or help, how can I optimize it

I have already searched previous posts but most of them did it in situation with enough light and contrast that is not applicable in this case

Answer 1

There are two global approaches we can take here: use computer vision all throughout (could be inaccurate), or use a workaround (simpler approach). The latter seems better. Thus, first, we have to crop the original image to be left with one that only has the grid of seedling plots. This can be done with any simple image processing tool (i.e., Preview on Mac or Paint on Windows):

Great, now, we have to identify the different squares. If we notice, we have a 10x6 grid of even squares, so we can use matplotlib as follows:

import matplotlib.pyplot as plt
import matplotlib.patches as patches

def draw_seedling_squares(image_path, grid_size=(10, 6)):
    # Load the image
    img = plt.imread(image_path)
    
    # Create a new figure
    fig, ax = plt.subplots()
    ax.imshow(img)
    
    # Calculate the width and height of each square
    img_height, img_width, _ = img.shape
    square_width = img_width / grid_size[0]
    square_height = img_height / grid_size[1]
    
    # Draw squares above each seedling plot
    for i in range(grid_size[0]):
        for j in range(grid_size[1]):
            x = i * square_width
            y = j * square_height
            rect = patches.Rectangle((x, y), square_width, square_height, linewidth=1, edgecolor='r', facecolor='none')
            ax.add_patch(rect)
    
    # Show the image with squares
    plt.show()

draw_seedling_squares('cropped_image.jpeg')

Which yields:

Very accurate! Now we have one task left: saving each seedling plot in a folder called screenshots, for example. To do this, we may modify our above code a little bit as follows:

import os
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.patches as patches

def extract_and_save_seedling_plots(image_path, output_folder='screenshots', grid_size=(10, 6)):
    # Create the output folder if it doesn't exist
    if not os.path.exists(output_folder):
        os.makedirs(output_folder)
    
    # Load the image
    img = plt.imread(image_path)
    
    # Calculate the width and height of each seedling plot
    img_height, img_width, _ = img.shape
    plot_width = img_width / grid_size[0]
    plot_height = img_height / grid_size[1]
    
    # Extract and save each seedling plot as individual files
    for i in range(grid_size[0]):
        for j in range(grid_size[1]):
            x = i * plot_width
            y = j * plot_height
            
            # Define the bounding box for cropping
            bbox = (int(x), int(y), int(x + plot_width), int(y + plot_height))
            
            # Crop the seedling plot
            seedling_plot = img[bbox[1]:bbox[3], bbox[0]:bbox[2]]
            
            # Save the cropped seedling plot as an individual file
            filename = os.path.join(output_folder, f'seedling_plot_{i}_{j}.png')
            plt.imsave(filename, seedling_plot)

extract_and_save_seedling_plots('cropped_image.jpeg')

Great! Now we have a folder with 60 cropped images like the ones below, corresponding to the first row:

Hope this helped! And may the code be with you...