Search code examples
pythonpandasappendrowunique-values

Pandas find whether cell has any value and if yes save the entire row to new dataframe

I have 284 dataframes where I need to iterate over the last 20 rows and if specific columns have any value then copy that entire row to a new dataframe. I get the desired result through the following:

 dataframes_list[0].tail(20).drop_duplicates(subset=["Buy","Sell"], keep=False)

By using a for loop, how can I save the result from all the 284 dataframes into a new dataframe?

Solution

  • You can use pandas concat function. If your line of code gives the correct result for one dataframe of the list, then the following code should work and the final result will be stored in the dataframe 'result':

    result = pd.DataFrame()
for i in range(284):
    df = dataframes_list[i].tail(20).drop_duplicates(subset=["Buy","Sell"], keep=False)
    result = pd.concat([result, df], ignore_index=True)