Search code examples
pythonnumpyimage-processingscikit-imagesmoothing

How to use skimage to denoise 2d array with nan values?


I'm trying to apply the TV filter to 2D array which includes many nan values:

from skimage.restoration import denoise_tv_chambolle
import numpy as np

data_random = np.random.random ([100,100])*100
plt.imshow(data_random)
plt.imshow(denoise_tv_chambolle(data_random))

data_random[20:30, 50:60] = np.nan
data_random[30:40, 55:60] = np.nan
data_random[40:50, 65:75] = np.nan

plt.imshow(denoise_tv_chambolle(data_random))

The TV filter works well with all valid data, but will return a nan array if there're nan values.

Original data:

Deonised data:

Data with nan values:


Solution

  • You can use a masked array:

    m = np.isnan(data_random)
    data = np.ma.masked_array(np.where(m, 0, data_random), m)
    plt.imshow(denoise_tv_chambolle(data, weight=50))
    

    Example output (with weight = 50):

    scikit denoise NaN masked array

    For less artifacts you could fill the holes with the average instead of zero:

    m = np.isnan(data_random)
    data = np.ma.masked_array(np.where(m, np.nanmean(data_random), data_random), m)
    plt.imshow(denoise_tv_chambolle(data, weight=50))
    

    Output:

    scikit denoise NaN masked array fill mean

    Another option would be to fill the holes with the nearest neighbors (e.g. with distance_transform_edt), then denoise, then restore the NaNs:

    from scipy.ndimage import distance_transform_edt
    
    m = np.isnan(data_random)
    data_fill = data_random[tuple(distance_transform_edt(m, return_distances=False, return_indices=True))]
    plt.imshow(np.where(m, np.nan, denoise_tv_chambolle(data_fill, weight=50)))
    

    Output:

    scikit denoise NaN fill nearest neighbors

    Intermediate data_fill:

    intermediated filled data