Can't find table position in image with opencv

Question

I want to process tabular data from images. To do this, we read the image with opencv and find where the table is by going through the following seven steps. In image number 7, we plan to crop based on the border. In the following example data, it works exactly what I want. This is because there is a black outer border of the image internal table.

image = cv2.imread(image_path, cv2.IMREAD_COLOR)

grayscaled_image = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)

_, thresholded_image = cv2.threshold(grayscaled_image, 0, 255, cv2.THRESH_BINARY | cv2.THRESH_OTSU)

inverted_image = cv2.bitwise_not(thresholded_image)

dilated_image = cv2.dilate(inverted_image, None, iterations=3)

contours, _ = cv2.findContours(dilated_image, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
image_with_all_contours = image.copy()
cv2.drawContours(image_with_all_contours, contours, -1, (0, 255, 0), 2)

rectangular_contours = []
for contour in contours:
    peri = cv2.arcLength(contour, True)
    epsilon = peri * 0.02
    approx = cv2.approxPolyDP(contour, epsilon, True)
    if len(approx) == 4:
        rectangular_contours.append(approx)
image_with_only_rectangular_contours = image.copy()
cv2.drawContours(image_with_only_rectangular_contours, rectangular_contours, -1, (0, 255, 0), 2)

max_area = 0
max_area_contour = None
for contour in rectangular_contours:
    area = cv2.contourArea(contour)
    if area > max_area:
        max_area = area
        max_area_contour = contour

image_with_max_area_contour = image.copy()
cv2.drawContours(image_with_max_area_contour, [max_area_contour], -1, (0, 255, 0), 2)

However, there are cases where the table does not have an outer border, as shown in the following picture. In reality, the image I want to work with does not have lines on the outside. The image below is a temporary work created for explanation purposes.

As you can see in the picture above, if there is no outer border, problems arise in the process of obtaining the Thresholded Image. Later, it becomes impossible to leave a square contour line by doing cv2.findContours.

Ultimately, what I want is to read the values in the Name and Favorite columns into Pandas. I am currently following the process by referring to this post. How can I select the rectangle of the largest contour line?

TRYING 1 by @Ivan

The following image is when cv2.RETE_EXTERNAL is applied when doing cv2.findContours. The cotour line is drawn as follows. That is, the outermost part is not drawn.

TRYING 2 by @cards

If you select 50*30 instead of the default kernel size, a distorted rectangle is selected as shown below.

dilated_image = cv2.dilate(inverted_image, np.ones((50, 30), np.uint8), iterations=1)

Answer 1

I'm not sure how general your example is, depending on this my answer might change.

To start:

%matplotlib notebook
import cv2
import numpy as np
import matplotlib.pyplot as plt
im = cv2.imread("table.jpg") # read im
b, g, r = cv2.split(im) # split to bgr
imRGB = cv2.merge([r,g,b]) # generate rgb image for plotting...

As a summary, it's always good to look at the different colours that make up an image. Here's how your image looks like in their RGB channels:

fig, axs = plt.subplots(nrows = 1, ncols = 3, sharex = True, sharey = True)
axs[0].imshow(r)
axs[1].imshow(g)
axs[2].imshow(b)
plt.suptitle("R|G|B")
plt.tight_layout()
for ax in axs:
    ax.axis("off")

We see from this that the R channel offers the most contrast between the table and the background.

So, thresholding there would make sense:

mask = (r<240).astype(np.uint8)
plt.figure()
plt.imshow(mask)
plt.axis("off")

We see here that indeed the threshold works nicely:

We can do a small closing operation and then find contours and draw contours on the image:

mask = (r<240).astype(np.uint8)
maskClosed = cv2.morphologyEx(mask, cv2.MORPH_CLOSE, np.ones((5,5)))
contours = cv2.findContours(maskClosed, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)[0]
largestContour = max(contours, key=cv2.contourArea)
imRGB = cv2.drawContours(imRGB, [largestContour], 0, (255,0,0), 5)
plt.figure()
plt.imshow(imRGB)
plt.axis("off")

We get the following:

Already quite good, we can use this with cv2.boundingRect(largestContour) to get the coordinates for cropping: (104, 68, 389, 137)

Let me know if this works for you or not..