Search code examples
pythontablefilter

Filter an existing dataframe to only include rows with a date value between two values in Python

I am trying to create a table from an existing table where only rows with date values between two numbers (43919 and 43989 aka March 29 and June 6 2020) are included. I'm doing this from an existing dataframe "county_timeseries_covid" and trying to create a new dataframe "filtered_table"

When I do the following, it creates a new table where all values are null except for those meeting the date requirement. But, I want the rows to be excluded rather than have null values:

filter1 = county_timeseries_covid["date"] >= 43919
filter2 = county_timeseries_covid["date"] <  43989

filtered_table = county_timeseries_covid.where(filter1 & filter2)

I've also tried the following, but this returns an empty table:

filter1 = county_timeseries_covid["date"] >= 43919
filter2 = county_timeseries_covid["date"] <  43989

filtered_table = county_timeseries_covid.where(filter1 & filter2, inplace = True)

I've also tried the following, but it returns an error:

filtered_table = county_timeseries_covid.where(county_timeseries_covid["date"] >= 43933 
                                                            , county_timeseries_covid["date"] < 44003)

How do I do this??? It seems so simple, but I can't find the answer despite a lot of googling.

Solution

  • "county_timeseries_covid" is a Pandas dataframe? If so, then the following should help you:

    county_timeseries_covid.loc[(county_timeseries_covid['date']>= 43919) & (county_timeseries_covid['date']< 43989) ,]