Search code examples
pythonlistnanextend

How to remove Nan's from a list and extend to a list of lists


I am trying to extend a list to a separate list of lists in it's correct order. But some of the values are Nan's. When I do this I get an error ValueError: could not convert string to float:. Would it be more efficient to iterate through the list removing Nan's and then extend to a list?

An example of the dataset and code is shown below:

Data:

X    Y
5    6
Nan  Nan
10   5
Nan  Nan
8    2

n=0
for column in data :
    if n == 0 :
        n+=1
        continue
    visuals[0].extend([float(column[0])])
    visuals[1].extend([float(column[1])])

Following on from the comments after @Mahesh Karia's code. The dummy data works fine but my dataset returns empty lists. An example of both is provided below:

data_1 = [['Nan', 5, 'Nan', 10, 'Nan', 8],
        ['Nan', 6, 'Nan', 5, 'Nan', 2]]

data_2 = [[nan, -5891.3764, -5901.0081390000005, -5939.977304, -5921.11797],
        [nan, 3339.025337, 3346.9211149999996, 3356.405148, 3412.836335]]

So data_1 works but data_2 returns an empty list?


Solution

  • def is_number(s):
        try:
            if str(s).lower() != "nan":
                float(s)
                return True
        except ValueError:
            pass
        return False
    
    
    data_2 = [['nan', -5891.3764, -5901.0081390000005, -5939.977304, -5921.11797],
            ['nan', 3339.025337, 3346.9211149999996, 3356.405148, 3412.836335]]
    
    visuals = [[],[]]
    
    visuals[0].extend([float(_) for _ in data_2[0] if is_number(_)])
    visuals[1].extend([float(_) for _ in data_2[1] if is_number(_)])
    
    print visuals
    

    output:

    [[-5891.3764, -5901.0081390000005, -5939.977304, -5921.11797], [3339.025337, 3346.9211149999996, 3356.405148, 3412.836335]]