Search code examples
pythonarraysnumpynan

How to filter arrays that contain nan out of an array of arrays


I know how to filter nan out of a simple array. But what if I have an array of arrays with some of them conatining nan, how do I filter them out?

Example:

arr = np.array([[1,2,3] [nan nan nan] [4,5,6]])

and I want the output to be:

[[1 2 3]
[4 5 6]]

furthermore I would like to know the indices where all the nan have been in the original array.

I tried using the following:

for i in range(len(arr)):
    if np.isnan(np.sum(arr[i])):
        arr=np.delete(arr, i, 0)

But unfortunately it doesn't work because the index will get out of range after deleting the desired subarrays.


Solution

  • You can do this in a vectorial way too. Here is an example including a row with both nan and non-nan values:

    import numpy as np
    
    arr = np.array([[1,2,3], [np.nan, np.nan, np.nan], [4,5,6], [7, np.nan, 8]])
    
    out = arr[ ~np.isnan(arr).any(axis=1) ]
    

    out:

    array([[1., 2., 3.],
           [4., 5., 6.]])