Search code examples
pythonpandasdataframecosine-similarityone-hot-encoding

How to aggregate row after one hot encoding


how to aggregate result after applying one hot encoding? below are my sample data

df= pd.DataFrame([
    ['apple','sweet'],
    ['apple','affordable'],
    ['apple','fruit'],
    ['orange','fruit'],
    ['orange','soup'],
    ['orange','cheap'],
    ['orange','sweet'],
    ['soda','sweet'],
    ['soda','cheap'],
    ['soda','softdrinks']
    ])

df= df.rename(columns={0: "productName", 1: "itemFeatures"})

i had tried

df_ohe = pd.get_dummies(df['itemFeatures'])
df_ohe_merged = pd.concat([df, df_ohe],axis='columns')
df_final = df_ohe_merged.drop(['itemFeatures'],axis='columns')

how to obtain desired output as below? or is there a better way?

desired_output = pd.DataFrame([
    ['apple',1,0,0,1,0,0,1],
    ['orange',0,1,0,1,0,1,1],
    ['soda',0,0,1,0,1,0,1]
])
desired_output = desired_output.rename(columns={0: "productName",
                                                1: "affordable",
                                                2: "cheap",
                                                3: "famous",
                                                4: "fruit",
                                                5: "softdrinks",
                                                6: "sour",
                                                7: "sweet",
                                               })

thanks a lot


Solution

  • Use pd.crosstab

    new_df = pd.crosstab(df['productName'],df['itemFeatures'],colnames = [None]).reset_index()
    

    Another approach is DataFrame.pivot_table

    new_df = (df.pivot_table(index = 'productName',
                             columns = 'itemFeatures',
                             aggfunc = 'size',
                             fill_value = 0)
                .reset_index()
                .rename_axis (columns = None))
    print(new_df)
    
      productName  affordable  cheap  fruit  softdrinks  soup  sweet
    0       apple           1      0      1           0     0      1
    1      orange           0      1      1           0     1      1
    2        soda           0      1      0           1     0      1