Search code examples
pandasindexingrowsdrop

How to Drop Pandas Rows Based on Index Condition (True/False)


I need to apply simply an index constructed from the condition of "NaN" from one dataframe "df1" to a different dataframe "df2" so that all the rows as indicated by "True" in the index are dropped from df1 and df2 dataframes.

the series rowswithnan index looks like this:

0   False
1   True
2   True
3   False

and df1 looks like this:

   plant_name  power_kwh  hour  day  month  year
0  AREC        32963.4    23   31     12    2020
1  AREC        35328.2    22   31     12    2020
2  AREC        37523.6    21   31     12    2020
3  AREC        36446.0    20   31     12    2020

and df2 looks like this:

   plant_name  power_obs_kwh  hour  day  month  year
0  AREC        31548.4        23   31     12    2020
1  AREC        34732.2        22   31     12    2020
2  AREC        39023.7        21   31     12    2020
3  AREC        34946.0        20   31     12    2020

Based on the "True" conditions of the index "rowswithnan", i need to delete or drop rows for both df1 and df2 such that they look like this for df2:

   plant_name  power_obs_kwh  hour  day  month  year
0  AREC        31548.4        23   31     12    2020
3  AREC        34946.0        20   31     12    2020

and df1, the same like this:

   plant_name  power_kwh  hour  day  month  year
0  AREC        32963.4    23   31     12    2020
3  AREC        36446.0    20   31     12    2020

I have tried something like this with an error:

df1.drop(rowswithnan,inplace=True,axis=1)

KeyError: '[False False False ... True True True] not found in axis'

thank you!


Solution

  • Boolean indexing is simpler here:

    df1 = df1[~rowswithnan]
    df2 = df2[~rowswithnan]
    

    It's also possible to use your original drop() method, but drop() wants the labels so we end up using boolean indexing anyway to get the index:

    df1.drop(df1[rowswithnan].index, inplace=True)
    df2.drop(df2[rowswithnan].index, inplace=True)