Is there a way to remove complex lines from an image with OpenCV?

Question

I have this 3 CAPTCHA images listed below and I want to make the text as clean as possible using opencv. Is there a pipeline that can achieve what I want?

Here is an example of what I want:

The biggest problem here is the color difference in letters. Any help would be appreciated!

Here are some parts from my code with all the image preprocessing

raw_img = cv2.imread(image_file) #load image
img = cv2.cvtColor(raw_img, cv2.COLOR_RGB2GRAY) #grayscale

_,thresh = cv2.threshold(img,240,255, cv2.THRESH_BINARY_INV + cv2.THRESH_TRUNC) #thresholding

thresh = cv2.bitwise_not(thresh) #invert black and white

#resizing image
resized = cv2.resize(img, (140, 60), interpolation=cv2.INTER_AREA)

img_gray = cv2.copyMakeBorder(resized, 10, 10, 10, 10, cv2.BORDER_CONSTANT, value=[255, 255, 255])

#blur
blur = cv2.GaussianBlur(img_gray, (3, 3), 0)

# threshold again
_, img = cv2.threshold(blur, 0, 255, cv2.THRESH_BINARY or cv2.THRESH_OTSU)

cv2.imshow("Output", img)
cv2.waitKey()

However, if I apply this same code to the second iamge, for example, here is what I get:

I'm relatively new to OpenCV. I´m not sure if I am using the right approach here, if someone can help me showing other methods to clean this images I´d be greatful!

Answer 1

As the background seems to be the same across images you can subtract the background to find the foreground objects. To obtain the background image you can run this code or use the image provided below.

import numpy as np
import cv2
img = cv2.imread('./test_img3.png')  # use the 3rd example
bg = np.repeat(img[:,0, np.newaxis], 180, axis=1)  # take the first column and repeat
cv2.imwrite('cap_bg.png', bg)

This gives the following image, which can then be used for subtraction:

To find the lines the Hough-Transformation can be used. It can find lines in greyscale images and is very well explained in the openCV documentation.

bin_img = ((img - bg) > 0).astype(np.uint8) * 255  # subtract background and obtain binary image
bin_img = cv2.cvtColor(bin_img, cv2.COLOR_RGB2GRAY)  # convert to greyscale image
lines = cv2.HoughLines(bin_img, 1, np.pi/360, 100)  # find lines

This output a list of all lines found in the image with the Hough transformation and the given parameters. Now you only have to 'erase' each line by drawing a line in the background color over them. You need to play around with the parameters, but I hope this can help for a start.