Search code examples
pythonpython-3.xpandasdataframecounter

How can I get mode(s) of pandas dataframe object values?


I have a pandas.DataFrame containing numerous columns. I am interested in just one of those columns ('names') whose type = 'object'. I want to answer three questions about this column:

  1. What value(s) appear most often excluding nan values?

  2. How many values meet that criteria (count of value in answer #1)?

  3. How often do those values appear?

I started with a large dataframe (df). The column I am interested in is called 'names'. First, I used collection.Counter to get the number of occurrences for each unique value in the 'names' column:

In [52]: cntr = collections.Counter([r for i, r in df['names'].dropna().iteritems()])
Out[52]: Counter({'Erk': 118,
    'James': 120,
    'John': 126,
    'Michael': 129,
    'Phil': 117,
    'Ryan': 126})

Then I converted the Counter back to a dataframe:

In [53]: df1 = pd.DataFrame.from_dict(cntr, orient='index').reset_index()
In [54]: df1 = df1.rename(columns={'index':'names', 0:'cnt'})

This gave me a pandas dataframe containing:

In [55]: print (type(df1), df1)
Out[55]: <class 'pandas.core.frame.DataFrame'>
       names    cnt
    0      Erk  118
    1    James  120
    2     Phil  117
    3     John  126
    4  Michael  122
    5     Ryan  126

The next part is where I need a bit of help. My desired output in this example is:

Answer #1 = [John, Ryan]

Answer #2 = 2

Answer #3 = 126

I am not convinced using the Counter was the best option, so I am open to options that stay within the dataframe without bouncing between dataframe to counter back to dataframe.


Solution

  • You can get that information directly from the Counter like:

    Code:

    from collections import Counter
    
    data = Counter({'Erk': 118, 'James': 120, 'John': 126,
                    'Michael': 122, 'Phil': 117, 'Ryan': 126})
    
    by_count = {}
    for k, v in data.items():
         by_count.setdefault(v, []).append(k)
    max_value = max(by_count.keys())
    print(by_count[max_value], len(by_count[max_value]), max_value)
    

    Results:

    ['John', 'Ryan'] 2 126