How can I contour low-contrast objects in python?
I am having difficulties contouring this type of low-contrast objects:
Where I aim for an output such as:
In the example above I used cv2.findContours with a code as the one below, but using a threshold value of 105 ret,thresh = cv.threshold(blur, 105, 255, 0). However, if I reproduce it for the low-contrast image, I fail to find an optimum threshold value:
import numpy as np
from PIL import Image
import requests
from io import BytesIO
import cv2 as cv
url = 'https://i.sstatic.net/OeZJ9.jpg'
response = requests.get(url)
img = Image.open(BytesIO(response.content)).convert('RGB')
img = np.array(img)
imgray = cv.cvtColor(img, cv.COLOR_BGR2GRAY)
blur = cv.GaussianBlur(imgray, (105, 105), 0)
ret,thresh = cv.threshold(blur, 205, 255, 0)
im2, cnts, hierarchy = cv.findContours(thresh,cv.RETR_TREE,cv.CHAIN_APPROX_SIMPLE)
cv.drawContours(img, cnts, -1, (0,0,255), 5)
plt.imshow(img, cmap = 'gray')
which outputs:
I understand that the problem is that the intensity of the background and the object overlap, but I can't find any other successful method. Other things I've tried include:
- Thresholding, in
skimagewithskimage.measure.find_contours. - Watershed algorithm, in
opencv. - Eroding and dilating in
opencv, which lowers too much the contour resolution.
I would appreciate help to contour, with as much resolution as possible, this object with low contrast respect to the background.
What has been proposed here
Contour detection by color gradient changes (See Antonino's reply)
Contouring objects with low contrast respect to the background is not a trivial task. Although Antonino's snippet gets close to contouring, it is not enough for contour detection:
The
finalContoursis not a single contour line, but an array of unclear lines, even if using the best possible parameters (see below):
To find the best possible parameters, I used the pseudocode below, which outputs thousands of images that were visually categorised (see output image). However, none of the combination of the possible parameters was successful, i.e. outputted the desired contour:
for scale_percent in range(30,51,5): for threshold1 in range(5, 21): for threshold2 in range(10,31): for gauss_kernel in range(1,11,2): for std in [0,1,2]: for kernel_size in range(2,6): for iterations_dialation in [2,3]: for iterations_erosion in [2,3]: for img in images: name = img[3:] img = cv2.imread('my/img/dir'+img) original_height, original_width, color = img.shape width = int(original_width * scale_percent / 100) height = int(original_height * scale_percent / 100) dim = (width, height) resized = cv2.resize(img, dim, interpolation = cv2.INTER_AREA) imgBlur = cv2.GaussianBlur(resized, (gauss_kernel, gauss_kernel), std) imgGray = cv2.cvtColor(imgBlur, cv2.COLOR_BGR2GRAY) imgCanny = cv2.Canny(imgGray, threshold1, threshold2) plt.subplot(231),plt.imshow(resized), plt.axis('off') plt.title('Original '+ str(name)) plt.subplot(232),plt.imshow(imgCanny,cmap = 'gray') plt.title('Canny Edge-detector\n thr1 = {}, thr2 = {}'.format(threshold1, threshold2)), plt.axis('off') kernel_s = (kernel_size, kernel_size) kernel = np.ones(kernel_s) imgDil = cv2.dilate(imgCanny, kernel, iterations = iterations_dialation) plt.subplot(233),plt.imshow(imgDil, cmap = 'gray'), plt.axis('off') plt.title("Dilated\n({},{}) iterations = {}".format(kernel_size, kernel_size, iterations_dialation)) kernel_erosion = np.ones(()) imgThre = cv2.erode(imgDil, kernel, iterations = iterations_erosion) plt.subplot(234),plt.imshow(imgThre, cmap = 'gray'), plt.axis('off') plt.title('Eroded\n({},{}) iterations = {}'.format(kernel_size, kernel_size, iterations_erosion)) imgFinalContours, finalContours = getContours(imgThre, resized) plt.subplot(235), plt.axis('off') plt.title("Contours") plt.subplot(236), plt.axis('off') plt.title('Contours') plt.tight_layout(pad = 0.1) plt.imshow(imgFinalContours) plt.savefig("my/results/" +name[:6]+"_scale_percent({})".format(scale_percent)+ "_threshold1({})".format(threshold1) +"_threshold2({})".format(threshold2) +"_gauss_kernel({})".format(gauss_kernel) +"_std({})".format(std) +"_kernel_size({})".format(kernel_size) +"_iterations_dialation({})".format(iterations_dialation) +"_iterations_erosion({})".format(iterations_erosion) +".jpg") plt.title(name) images = ["b_36_2.jpg", "b_78_2.jpg", "b_51_2.jpg","b_72_2.jpg", "a_78_2.jpg", "a_70_2.jpg"] process_images_1(images)
which outputs:
The solution
Using a pretrained Deep Learning model
A preliminary idea was to use grabcut to train a model, but that would be very costly in terms of time. Therefore, pretrained Deep Learning models were the first shot. While some tools failed, this other tool outperformed any other method tried before (see image below). Hence, all the credit to the creator of the GitHub repository, extended to the creators of operating models (U^2-NET, BASNet). The https://github.com/OPHoperHPO/image-background-remove-tool doesn't need any image preprocessing, contains a very straightforward documentation on how to deploy it, and even an executable google colab notebook. The output image is a png image with transparent background:
Hence, all it takes to find the contour is to isolate the alpha channel:
import cv2
import matplotlib.pyplot as plt, numpy as np
filename = '/a_58_2_pg_0.png'
image_4channel = cv2.imread(filename, cv2.IMREAD_UNCHANGED)
alpha_channel = image_4channel[...,-1]
contours, hier = cv2.findContours(alpha_channel, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
for idx,contour in enumerate(contours):
# create mask
# zeros with same shape
mask = np.zeros(alpha_channel.shape,np.uint8)
# draw contour
mask = cv2.drawContours(mask,[contour],-1,(255,255,255),-1) # -1 to fill the mask
cv2.imwrite('/contImage.jpg', mask)
plt.imshow(mask)
