Search code examples
pythonnumpyscipynoise-reduction

Noise removal in spatial data

I have an array with two different values that characterize this image: enter image description here

I would like to keep the linear trends in red and remove the "noise" (single red points). Is there a good way to do this?

Solution

  • If there is no way to determine the noise from the signal based on a threshold value (i.e. all the red point have the same value or are just a 1/0 flag), a relatively simple but easy to implement approach could be to look at removing the noise based on the size of the clumps.

    Take a look at scipy's label. This will give you an array where each separate 'clump' has an individual number. It is then a case of just removing those features that are less then some threshold number of pixels (n_thresh below).

    >>> from scipy.ndimage.measurements import label
>>> import numpy as np
>>> n_thresh = 1
>>> a = np.array([[0,0,1,1,0,0],[0,0,0,1,0,0],
                  [1,1,0,0,1,0],[0,0,0,1,0,0],
                  [1,1,0,0,1,0],[0,0,1,1,0,0]])
>>> a
array([[0, 0, 1, 1, 0, 0],
       [0, 0, 0, 1, 0, 0],
       [1, 1, 0, 0, 1, 0],
       [0, 0, 0, 1, 0, 0],
       [1, 1, 0, 0, 1, 0],
       [0, 0, 1, 1, 0, 0]])
>>> labeled_array, num_features = label(a)
>>> binc = np.bincount(labeled_array.ravel())
>>> noise_idx = np.where(binc <= n_thresh)
>>> shp = a.shape
>>> mask = np.in1d(labeled_array, noise_idx).reshape(shp)
>>> a[mask] = 0
>>> a
array([[0, 0, 1, 1, 0, 0],
       [0, 0, 0, 1, 0, 0],
       [1, 1, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 0],
       [1, 1, 0, 0, 0, 0],
       [0, 0, 1, 1, 0, 0]])
>>> a

    As the features are diagonal, you may want to pay attention to the example in label's documentations that groups diagonally-touching pixels in the sample clump.