Python: How to cut out an area with specific color from image (OpenCV, Numpy)

Question

so I've been trying to code a Python script, which takes an image as input and then cuts out a rectangle with a specific background color. However, what causes a problem for my coding skills, is that the rectangle is not on a fixed position in every image (the position will be random).

I do not really understand how to manage the numpy functions. I also read something about OpenCV, but I'm totally new to it. So far I just cropped the images through the ".crop" function, but then I would have to use fixed values.

This is how the input image could look and now I would like to detect the position of the yellow rectangle and then crop the image to its size.

Help is appreciated, thanks in advance.

Edit: @MarkSetchell's way works pretty good, but found a issue for a different test picture. The problem with the other picture is that there are 2 small pixels with the same color at the top and the bottom of the picture, which cause errors or a bad crop.

Answer 1

Updated Answer

I have updated my answer to cope with specks of noisy outlier pixels of the same colour as the yellow box. This works by running a 3x3 median filter over the image first to remove the spots:

#!/usr/bin/env python3

import numpy as np
from PIL import Image, ImageFilter

# Open image and make into Numpy array
im = Image.open('image.png').convert('RGB')
na = np.array(im)
orig = na.copy()    # Save original

# Median filter to remove outliers
im = im.filter(ImageFilter.MedianFilter(3))

# Find X,Y coordinates of all yellow pixels
yellowY, yellowX = np.where(np.all(na==[247,213,83],axis=2))

top, bottom = yellowY[0], yellowY[-1]
left, right = yellowX[0], yellowX[-1]
print(top,bottom,left,right)

# Extract Region of Interest from unblurred original
ROI = orig[top:bottom, left:right]

Image.fromarray(ROI).save('result.png')

Original Answer

Ok, your yellow colour is rgb(247,213,83), so we want to find the X,Y coordinates of all yellow pixels:

#!/usr/bin/env python3

from PIL import Image
import numpy as np

# Open image and make into Numpy array
im = Image.open('image.png').convert('RGB')
na = np.array(im)

# Find X,Y coordinates of all yellow pixels
yellowY, yellowX = np.where(np.all(na==[247,213,83],axis=2))

# Find first and last row containing yellow pixels
top, bottom = yellowY[0], yellowY[-1]
# Find first and last column containing yellow pixels
left, right = yellowX[0], yellowX[-1]

# Extract Region of Interest
ROI=na[top:bottom, left:right]

Image.fromarray(ROI).save('result.png')

---

You can do the exact same thing in Terminal with ImageMagick:

# Get trim box of yellow pixels
trim=$(magick image.png -fill black +opaque "rgb(247,213,83)" -format %@ info:)

# Check how it looks
echo $trim
251x109+101+220

# Crop image to trim box and save as "ROI.png"
magick image.png -crop "$trim" ROI.png

If still using ImageMagick v6 rather than v7, replace magick with convert.