Search code examples
pythonopencvimage-processingfeature-extractionlbph-algorithm

Apply object mask to LBP calculation

I see lots of articles applying lbp for texture based image classification. I just wonder three things about this technique that I couldn't find clear answers from Google:

  1. How the algorithm calculates lbp for border pixels of an image that don't have enough neighbor pixels around them.

  2. If we have eight neighbor pixels then the central pixel will have 256 patterns (59 if using uniform). But if we increase the neighbor pixel size (e.g. 8 or 10), then number of patterns will also increase, is that right? In that case, how it impacts to histogram calculation?

  3. How can we calculate lbp for the object only. Particularly, if we want to compare objects in images, we only need to calculate lbp and histogram for the objects. I have tried to this idea by using opencv histogram (which supports mask and numpy histogram doesn't support mask) to lbp output but it didn't work. Any idea about how to filter lbp array based on mask and then we can find histogram afterward.

Thank you.

Solution

    1. Border pixels are usually discarded (have a look at the Matlab implementation developed by the research group that first proposed LBP). In other implementations (see for example Python's scikit-learn) a black border is added though.

    2. A local neighborhood of P pixels results in a 2P-bin histogram. If you set P=10, then the feature vector would have 1024 components.

    3. This toy example shows you how to filter out unwanted image regions from the LBP histogram calculation:

      import numpy as np
from skimage.feature.texture import local_binary_pattern

P, R = 8, 1
dim = 2**P
img = np.asarray([[5, 5, 5, 5], [5, 99, 100, 5], [5, 5, 5, 5]]], dtype=np.uint8)
mask = img > 5
codes = local_binary_pattern(img, P, R)
hist, _ = np.histogram(codes[mask], bins=np.arange(dim+1), range=(0, dim))

      Demo

      In [97]: img
Out[97]: 
array([[  5,   5,   5,   5],
       [  5,  99, 100,   5],
       [  5,   5,   5,   5]], dtype=uint8)

In [98]: codes
Out[98]: 
array([[ 193.,  241.,  241.,  112.],
       [ 199.,    1.,    0.,  124.],
       [   7.,   31.,   31.,   28.]])

In [99]: mask
Out[99]: 
array([[False, False, False, False],
       [False,  True,  True, False],
       [False, False, False, False]], dtype=bool)

In [100]: hist
Out[100]: 
array([1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
       0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
       0, 0, 0], dtype=int32)

    EDIT

    Here's a more realistic example, as you requested in the comments:

    import numpy as np
from skimage import io
from skimage.feature.texture import local_binary_pattern
import matplotlib.pyplot as plt

P = 8
R = 1
dim = 2**P
h_bins = np.arange(dim+1)
h_range = (0, dim)

img = io.imread('https://i.sstatic.net/6ESoP.png')
mask = (img > 0)
codes = local_binary_pattern(img, P, R)
h_img, _ = np.histogram(codes.ravel(), bins=h_bins, range=h_range)
h_masked, _ = np.histogram(codes[mask], bins=h_bins, range=h_range)
h_img = h_img/h_img.sum(dtype=np.float)
h_masked = h_masked/h_masked.sum(dtype=np.float)

f, [[ax0, ax1], [ax2, ax3]] = plt.subplots(2, 2)
ax0.imshow(img, cmap=plt.cm.gray)
ax0.axis('off')
ax0.set_title('Image')
ax1.imshow(mask, cmap=plt.cm.gray)
ax1.axis('off')
ax1.set_title('Mask')
ax2.plot(h_img)
ax2.set_title('LBP of image')
ax3.plot(h_masked)
ax3.set_title('LBP of ROI')
plt.show(f)

    Results