Search code examples
pythonpandasnankeyerror

isna search on mulitple columns/rows gives error: duplicate columns & possible reduce dimensionality


I'm trying to search for NaN values in different columns and rows and want to show all rows.

My dataset:

import pandas as pd
df = pd.read_excel('./file.xlsx')
df = df.replace(' ', np.nan)
df.head()
 
                             col1    col2      col3

row1                         NaN     NaN       test
row2                         hello   twon      street
row3                         words   place     NaN
row4                         NaN     towns     places
row5                         town    NaN       NaN  

I tried this:

nan_rows = df[df['col1', 'col2', 'col3'].isna().any(axis=1)]

This gives a key-error:

  2693         # get column
   2694         if self.columns.is_unique:
-> 2695             return self._get_item_cache(key)
   2696 
   2697         # duplicate columns & possible reduce dimensionality

Expected result:

                             col1    col2      col3

row1                         NaN     NaN       test
row3                         words   place     NaN
row4                         NaN     towns     places
row5                         town    NaN       NaN

Appreciate the help and effort. Thank you!


Solution

  • You need double square brackets to slice with a list of columns:

    df[df[['col1', 'col2', 'col3']].isna().any(axis=1)]
    

    Alternative using a named list:

    cols = ['col1', 'col2', 'col3']
    
    df[df[cols].isna().any(axis=1)]
    

    output:

           col1   col2    col3
    row1    NaN    NaN    test
    row3  words  place     NaN
    row4    NaN  towns  places
    row5   town    NaN     NaN