How to detect a specific border detail of an image and crop according to it using python and PIL?

Question

I'm currently trying to crop an image by detecting the 2 big black corners on the upper-right and lower-left pf the document: http://www.suiteexpert.fr/AideProd/SalaireExpert/Images/Editions/aemapercu.png I'm trying to find a simpler method than deep-learning OCR for better performance.

I have tried a few different things such as detecting a change in color, by starting along the edges of the document and detecting a change from white to black. This seems to work but it first detects the very small vertical text on some of the image examples: https://www.movinmotion.com/wp-content/uploads/2018/03/AEM-V5-MM-.jpg

This also leeds to other problems due to the fact that the documents I want to analyze could also be in different colors, and could also be scans or photographs.

However, these documents will always have the same internal shape and structure (it's a standardized administrative document).

I would want some of your feedback as to how I could detect the 2 big black corners on the upper right and lower left corners of the document, which would allow me process and normalize the image by rotating and cropping it.

from PIL import Image, ImageChops

def trim(im, border):
     bg = Image.new(im.mode, im.size, border)
     diff = ImageChops.difference(im, bg)
     bbox = diff.getbbox()
     if bbox:
         return im.crop(bbox)
     else:
         # found no content
         raise ValueError("cannot trim; image was empty")

Thank you in advance for your help!

Answer 1

I haven't got time to write this up in Python at the moment, but if you follow these steps you can do it:

• do a "morphological closing" with a rectangular 5x5 structuring element to get rid of the thin lines around the edge (and everywhere else)

• trim the image to get rid of excess white borders around the edges

I did it with ImageMagick in Terminal like this:

convert form.png -threshold 50% -morphology close rectangle:5 -trim result.png

and then sneakily added a red border just so you can see the extent of the image on Stack Overflow's white background:

In Python, use skimage to do the morphology - it'll look like these lines:

# Convert edges to Numpy array and dilate (fatten) with our square structuring element
selem = square(6)
fatedges = dilation(np.array(edges),selem)

in this answer, except you will need binary_closing() in place of dilation() - see documentation here.

You can then do the trimming with PIL/Pillow's Image.getbbox(). Of course, you apply the trim box to a clean copy of your original image, not the morphologically altered one :-)