Python OpenCV template matching and feature detection not working properly
I'm attempting to identify specific shapes in an image using a template. I've edited the original image by adding two stars to it. Now, I'm trying to detect the positions of these stars, but it doesn't seem to be recognizing them. I've employed two methods, template matching and feature detection, but neither is yielding the expected results.
My primary objective is to successfully detect both stars and connect them to form a rectangular shape, ultimately providing me with just the answers area as a result in a new image. However, my initial focus is on accurately detecting the marks.
The base image and the template are as follows:
First i've tried to match the image with the template.
# Load the base image and the black star image
base_image = cv2.imread(path_saida_jpg, cv2.IMREAD_COLOR)
star_template = cv2.imread(path_base_star, cv2.IMREAD_COLOR)
base_gray = cv2.cvtColor(base_image, cv2.COLOR_BGR2GRAY)
star_template_gray = cv2.cvtColor(star_template, cv2.COLOR_BGR2GRAY)
# Perform template matching to find the stars
result = cv2.matchTemplate(base_image, star_template, cv2.TM_CCOEFF_NORMED)
threshold = 0.80
locations = np.where(result >= threshold)
# Draw rectangles around the found stars
for loc in zip(*locations[::-1]):
h, w = star_template.shape[:2]
cv2.rectangle(base_image, loc, (loc[0] + w, loc[1] + h), (0, 255, 0), 2) # Draw a green rectangle
cv2.imwrite('result_image.jpg', base_image)
The locations were always empty, the software as only capable to find some match when i down the threshold variable to 0.6. but the result wasn't the expected. Here it is:
Afterward, I realized that the issue might be related to scale. Consequently, I attempted to address it using feature detection and matching with ORB (Oriented FAST and Rotated BRIEF). In my initial attempt, I obtained no results when using the template image at its original scale, which was 64px. Consequently, I decided to increase its size to 512px, hoping to enhance the chances of success. Surprisingly, the outcome was even less favorable. Below is the code and the resulting output:
base_image = cv2.imread(path_saida_jpg, cv2.IMREAD_COLOR)
star_template = cv2.imread(path_base_star, cv2.IMREAD_COLOR)
base_gray = cv2.cvtColor(base_image, cv2.COLOR_BGR2GRAY)
star_template_gray = cv2.cvtColor(star_template, cv2.COLOR_BGR2GRAY)
orb = cv2.ORB_create()
kp1, des1 = orb.detectAndCompute(base_gray, None)
kp2, des2 = orb.detectAndCompute(star_template_gray, None)
bf = cv2.BFMatcher(cv2.NORM_HAMMING, crossCheck=True)
matches = bf.match(des1, des2)
matches = sorted(matches, key=lambda x: x.distance)
result_image = cv2.drawMatches(base_image, kp1, star_template, kp2, matches[:10], None, flags=cv2.DrawMatchesFlags_NOT_DRAW_SINGLE_POINTS)
# Export the result
cv2.imwrite('result_image.jpg', result_image)
Does anyone have any suggestions? I'm relatively new to the field of image processing, and I find myself somewhat disoriented in the midst of this complex task. I've been using ChatGPT and resources from GeeksforGeeks to assist me.
I have cropped your star from your image to use as a template in Python/OpenCV template matching. I then use a technique of masking the correlation image in a loop to find the two matches. Each top match is masked out with zeros in the TM_CCORR_NORMED (normalized cross correlation) surface before searching for the next top match.
- Read the input
- Read the template
- Set arguments
- Compute TM_CCORR_NORMED image
- Loop over matches, if max_val > match_threshold, save correlation location and value and mask the correlation image. Then repeat until max_val below match_thresh.
- Save results
Input:
Template:
import cv2
import numpy as np
# read image
img = cv2.imread('prova_da_epcar.jpg')
# read template
tmplt = cv2.imread('star.png')
hh, ww, cc = tmplt.shape
# set arguments
match_thresh = 0.95 # stopping threshold for match value
num_matches = 10 # stopping threshold for number of matches
match_radius = max(hh,ww)//4 # approx radius of match peaks
# get correlation surface from template matching
corrimg = cv2.matchTemplate(img,tmplt,cv2.TM_CCORR_NORMED)
hc, wc = corrimg.shape
# get locations of all peaks higher than match_thresh for up to num_matches
imgcopy = img.copy()
corrcopy = corrimg.copy()
for i in range(1, num_matches):
# get max value and location of max
min_val, max_val, min_loc, max_loc = cv2.minMaxLoc(corrcopy)
x1 = max_loc[0]
y1 = max_loc[1]
x2 = x1 + ww
y2 = y1 + hh
loc = str(x1) + "," + str(y1)
if max_val > match_thresh:
print("match number:", i, "match value:", max_val, "match x,y:", loc)
# draw draw blue bounding box to define match location
cv2.rectangle(imgcopy, (x1,y1), (x2,y2), (0,0,255), 2)
# draw black filled circle over copy of corrimg
cv2.circle(corrcopy, (x1,y1), match_radius, 0, -1)
# faster alternate - insert black rectangle
# corrcopy[y1:y2, x1:x2] = 0
i = i + 1
else:
break
# save results
# power of 4 exaggeration of correlation image to emphasize peaks
cv2.imwrite('prova_da_epcar_corr.png', (255*cv2.pow(corrimg,4)).clip(0,255).astype(np.uint8))
cv2.imwrite('prova_da_epcar_star_corr_masked.png', (255*cv2.pow(corrcopy,4)).clip(0,255).astype(np.uint8))
cv2.imwrite('prova_da_epcar_star_multi_match.png', imgcopy)
# show results
# power of 4 exaggeration of correlation image to emphasize peaks
cv2.imshow('image', img)
cv2.imshow('template', tmplt)
cv2.imshow('corr', cv2.pow(corrimg,4))
cv2.imshow('corr masked', cv2.pow(corrcopy,4))
cv2.imshow('result', imgcopy)
cv2.waitKey(0)
cv2.destroyAllWindows()
Correlation Image:
Final Masked Correlation Image:
Result:
I suggest in the future that you use 3 or 4 symbols in your image to help if your image is slightly rotated. You might also consider using a ring as your symbol to be rotation independent.
I also suggest that you use cv2.TM_CCORR_NORMED (or cv2.TM_SQDIFF_NORMED and mask with white) rather than cv2.TM_CCOEF_NORMED
There are also non-maxima suppression methods for detecting multiple matches that do not need masking. See this example.