Search code examples
pythonpandasnumpyqlikviewqliksense

Counting unique values in a column in pandas dataframe like in Qlik?


If I have a table like this:

df = pd.DataFrame({
         'hID': [101, 102, 103, 101, 102, 104, 105, 101],
         'dID': [10, 11, 12, 10, 11, 10, 12, 10],
         'uID': ['James', 'Henry', 'Abe', 'James', 'Henry', 'Brian', 'Claude', 'James'],
         'mID': ['A', 'B', 'A', 'B', 'A', 'A', 'A', 'C']
})

I can do count(distinct hID) in Qlik to come up with count of 5 for unique hID. How do I do that in python using a pandas dataframe? Or maybe a numpy array? Similarly, if were to do count(hID) I will get 8 in Qlik. What is the equivalent way to do it in pandas?


Solution

  • Count distinct values, use nunique:

    df['hID'].nunique()
    5
    

    Count only non-null values, use count:

    df['hID'].count()
    8
    

    Count total values including null values, use the size attribute:

    df['hID'].size
    8
    

    Edit to add condition

    Use boolean indexing:

    df.loc[df['mID']=='A','hID'].agg(['nunique','count','size'])
    

    OR using query:

    df.query('mID == "A"')['hID'].agg(['nunique','count','size'])
    

    Output:

    nunique    5
    count      5
    size       5
    Name: hID, dtype: int64