Search code examples
pythonpandascountfrequency

Count appearances of a value until it changes to another value


I have the following DataFrame:

df = pd.DataFrame([10, 10, 23, 23, 9, 9, 9, 10, 10, 10, 10, 12], columns=['values'])

I want to calculate the frequency of each value, but not an overall count - the count of each value until it changes to another value.

I tried:

df['values'].value_counts()

but it gives me

10    6
9     3
23    2
12    1

The desired output is

10:2 
23:2
 9:3
10:4
12:1

How can I do this?


Solution

  • Use:

    df = df.groupby(df['values'].ne(df['values'].shift()).cumsum())['values'].value_counts()
    

    Or:

    df = df.groupby([df['values'].ne(df['values'].shift()).cumsum(), 'values']).size()
    

    print (df)
    values  values
    1       10        2
    2       23        2
    3       9         3
    4       10        4
    5       12        1
    Name: values, dtype: int64
    

    Last for remove first level:

    df = df.reset_index(level=0, drop=True)
    print (df)
    values
    10    2
    23    2
    9     3
    10    4
    12    1
    dtype: int64
    

    Explanation:

    Compare original column by shifted with not equal ne and then add cumsum for helper Series:

    print (pd.concat([df['values'], a, b, c], 
                     keys=('orig','shifted', 'not_equal', 'cumsum'), axis=1))
        orig  shifted  not_equal  cumsum
    0     10      NaN       True       1
    1     10     10.0      False       1
    2     23     10.0       True       2
    3     23     23.0      False       2
    4      9     23.0       True       3
    5      9      9.0      False       3
    6      9      9.0      False       3
    7     10      9.0       True       4
    8     10     10.0      False       4
    9     10     10.0      False       4
    10    10     10.0      False       4
    11    12     10.0       True       5