Search code examples
pythonpandasfilterfilteringfrequency

Filtering with two conditions - Remove duplicates less than a certain value while keeping the original


I have a table

total type
23 Original
3 Duplicate
11 Duplicate
5 Original
16 Duplicate
4 Duplicate

I want to Filter the df['total'] column for only values greater than 10, however, I want to remove only the Duplicates less or equal than 10. So if an Original row is less than 10 it can still be in the df.

This is my Desirable output:

total type
23 Original
11 Duplicate
5 Original
16 Duplicate

I tried this:

df[(df['total'] > 10) & df['type'] == "Duplicate"]

is not working.

Any idea?


Solution

  • The conditions should be enclosed in parentheses, on the right you have square ones. And to get what you showed. You need to add a condition(df['type'] =="Original"), in my opinion.

    a = df[(df['total'] > 10) & (df['type'] == "Duplicate")|(df['type'] == "Original")]
    print(a)
    

    Output a

       total       type
    0     23   Original
    2     11  Duplicate
    3      5   Original
    4     16  Duplicate