Create Bounding Boxes from Image Labels

Question

I have a 2D color image and a label image (projection of labels)

The output of label image is as follows:

[[16 16 16 ... 16 16 16 ]
 [16 16 16 ... 16 16 16 ]
 [16 16 16 ... 16 16 16 ]
 ...
 [ 2  2  2 ...  2  2  2 ]
 [ 2  2  2 ...  2  2  2 ]
 [ 2  2  2 ...  2  2  2 ]]

How do I draw bounding boxes around all the objects (represented by labels) in the original 2D color image?

Answer 1

This task can be easily done using NumPy's boolean array indexing and OpenCV's boundingRect function.

From here, I took this image

and this segmentation mask

The mask is an indexed image, which OpenCV has problems with, see also here. That's why, we'll also use Pillow for this task.

Here's the code:

import cv2
import numpy as np
from PIL import Image

# Read color image
img = cv2.imread('0.jpg')

# Read mask; OpenCV can't handle indexed images, so we need Pillow here
# for that, see also: https://stackoverflow.com/q/59839709/11089932
mask = np.array(Image.open('0_mask.png'))

# Iterate all colors in mask
for color in np.unique(mask):

    # Color 0 is assumed to be background or artifacts
    if color == 0:
        continue

    # Determine bounding rectangle w.r.t. all pixels of the mask with
    # the current color
    x, y, w, h = cv2.boundingRect(np.uint8(mask == color))

    # Draw bounding rectangle to color image
    out = cv2.rectangle(img.copy(), (x, y), (x+w, y+h), (0, int(color), 0), 2)

    # Show image with bounding box
    cv2.imshow('img_' + str(color), out)

# Show mask
cv2.imshow('mask', mask)
cv2.waitKey(0)
cv2.destroyAllWindows()

These are the outputs:

I decided to output several images, because for the given image, the bounding boxes heavily overlap. To draw all rectangles to the same image, just replace the corresponding rectangle command by

img = cv2.rectangle(img, (x, y), (x+w, y+h), (0, int(color), 0), 2)

----------------------------------------
System information
----------------------------------------
Platform:    Windows-10-10.0.16299-SP0
Python:      3.8.5
NumPy:       1.19.2
OpenCV:      4.4.0
Pillow:      7.2.0
----------------------------------------