Search code examples
pythonpandaslistnonetype

Filtering out None from the list of pd.Series / or finding out Nonetype objects in the list of pd.Series

I'm having a problem while removing Nonetype objects in the list. ValueError: The truth value of a Series is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all().

This is incurred when:

import pandas as pd
#List of pd.Series with some of the objects are None
results = [pd.Series([1,2,3,4]), None, pd.Series([2,3,4,5])]
print(list(filter(None, results)))

How can I filter out the Nonetype objects in the list? I know I can do with loops, but it'll not be pythonic ways.

Solution

  • When the filterfunction is None, python will do a truth test on each item (bool(item)). So, when should a pd.Series be True? Is it when all of its values are True? Any of the values? Maybe just when the series is not empty. The point is, there are many ways a series could be true. That's what the error message is telling you.

    Since your goal is to filter out None, and list comprehensions are a better choice than filter anyway, you can just test for None.

    print([value for value in results if value is not None])