How to identify black border of photo and remove this border?

Question

I need to detect black border in image. I tried to detect it using OpenCV, but the results were not good. Code here:

import cv2
import numpy as np
import matplotlib.pyplot as plt
from skimage.filters import threshold_local

image = cv2.imread("image.jpg")

V = cv2.split(cv2.cvtColor(image, cv2.COLOR_BGR2HSV))[2]
T = threshold_local(V, 15, offset=5, method="gaussian")
thresh = (V > T).astype("uint8") * 255
contours, _ = cv2.findContours(thresh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cv2.drawContours(image, contours, -1, (0, 255, 0), 2)
plt.imshow(thresh, cmap='gray')

Other code:

def find_threshold_img(img):
    gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
    # # apply thresholding to convert grayscale to binary image
    # ret,thresh = cv2.threshold(gray,100,200,0)
    thresh = cv2.threshold(gray, 0, 30, cv2.THRESH_BINARY|cv2.THRESH_OTSU)[1]
    return thresh
thresh = find_threshold_img(image)
contours, _ = cv2.findContours(thresh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cv2.drawContours(image, contours, -1, (0, 255, 0), 2)

plt.imshow(image, cmap='gray')

This is the image:

Answer 1

The basic idea is to detect everything that is not black. You can HSV-threshold on black, apply some morphology to get a nice binary mask, detect external contours and keep the biggest contour in the frame. Finally, get the bounding rectangle of this contour and use numpy slicing to crop the original image.

# Imports:
import numpy as np
import cv2

# Set image path
imagePath = "D://opencvImages//2fzmatjM.jpg"

# Load image:
inputImage = cv2.imread(imagePath)
# Deep copy of the input:
inputImageCopy = inputImage.copy()

# Resize because the original image is HUGE:
(h, w) = inputImage.shape[:2]
resizePercent = 10
resizedWidth = int(w * resizePercent / 100)
resizedHeight = int(h * resizePercent / 100)

# Resize image
inputImage = cv2.resize(inputImage, (resizedWidth, resizedHeight), interpolation=cv2.INTER_LINEAR)

# Get binary image:
grayImage = cv2.cvtColor(inputImage, cv2.COLOR_BGR2HSV)

lowerValues = np.array([0, 0, 40])
upperValues = np.array([179, 255, 255])

# Create HSV mask:
hsvImage = cv2.cvtColor(inputImage.copy(), cv2.COLOR_BGR2HSV)
tempMask = cv2.inRange(hsvImage, lowerValues, upperValues)

# Apply some morphology:
kernelSize = (3, 3)
morphoKernel = cv2.getStructuringElement(cv2.MORPH_RECT, kernelSize)
tempMask = cv2.morphologyEx(tempMask, cv2.MORPH_CLOSE, morphoKernel, iterations=1)

This will produce the following binary mask:

Then, just detect the biggest contour and keep its bounding rectangle. Also, don't forget to upscale the rectangle to crop the original image:

# Find the largest external contour:
contours, _ = cv2.findContours(tempMask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)

# Area filter:
maxArea = 0
maxRectangle = ()

for c in contours:

    # Blob Area:
    blobArea = cv2.contourArea(c)

    # Keep largest blob only:
    if blobArea > maxArea:
        # Set max Area:
        maxArea = blobArea

        # Get the bounding rectangle:
        boundRect = cv2.boundingRect(c)

        # Get the dimensions of the bounding rect:
        rectX = int(boundRect[0])
        rectY = int(boundRect[1])
        rectWidth = int(boundRect[2])
        rectHeight = int(boundRect[3])

        # Into a nice tuple:
        maxRectangle = (rectX, rectY, rectWidth, rectHeight)

# Upscale rectangle:
upscaledRectangle = []
for dimension in maxRectangle:
    upscaledDimension = int(dimension * 100 / resizePercent)
    upscaledRectangle.append(upscaledDimension)

# Crop image:
croppedImage = inputImageCopy[upscaledRectangle[1]:upscaledRectangle[1] + upscaledRectangle[3],
               upscaledRectangle[0]:upscaledRectangle[0] + upscaledRectangle[2]]

Which produces the final image. Again, resized, because the image is gargantuan:

Note: The border is not straight. In fact, this looks like a photo of a monitor and the black border isn't really solid black. Also, you are mixing different image processing libraries which is not great because they are not compatible with each other.