Search code examples
python-3.xdataframeboolean-logic

negating pandas isna function returning -2


I am a bit confused by inconsistency of how pd.isna() work in different settings. Any thoughts around why boolean not ~ doesn't work with pd.isna in number settings:

>>> import pandas as pd
>>> import numpy as np
>>> df = pd.DataFrame({'p':['a', 'b', 'c'], 'q':[1, np.nan, 2]})
>>> df
   p    q
0  a  1.0
1  b  NaN
2  c  2.0
>>> df[~pd.isna(df['q'])]       # THIS WORKS AS EXPECTED.
   p    q
0  a  1.0
2  c  2.0
>>> pd.isna(np.nan)
True
>>> not pd.isna(np.nan)
False                           # THIS WORKS AS EXPECTED.
#### So far so good ####
>>> ~pd.isna(np.nan)
-2                              # I WAS EXPECTING False HERE.
>>> ~(pd.isna(np.nan))
-2                              # I WAS EXPECTING False HERE.

I am not why last two rows fetch -2 instead of False.


Solution

  • This is not about pd.isna() in the last two rows you use ~ that is Bitwise Not Operator and it inverts all the bits of an object and binary not for True(1 in int) is -2:

    >>> var = int(True)
    >>> var
    1
    >>> not var
    False
    >>> ~ var
    -2
    

    Boolean values like True and False in python are return from comparison, logical and membership operators in python and bitwise operators return int values.