Search code examples
pythonpandasdataframeone-hot-encodingfeature-engineering

How to create by default two columns for every features (One Hot Encoding)?


My feature engineering runs for different documents. For some documents some features do not exist and followingly the sublist consists only of the same values such as the third sublist [0,0,0,0,0]. One hot encoding of this sublist leads to only one column, while the feature lists of other documents are transformed to two columns. Is there any possibility to tell ohe also to create two columns if it consits only of one and the same value and insert the column in the right spot? The main problem is that my feature dataframe of different documents consists in the end of a different number of columns, which make them not comparable.

import pandas as pd 
feature = [[0,0,1,0,0], [1,1,1,0,1], [0,0,0,0,0], [1,0,1,1,1], [1,1,0,1,1], [1,0,1,1,1], [0,1,0,0,0]]

df = pd.DataFrame(feature[0])
df_features_final  = pd.get_dummies(df[0])

for feature in feature[1:]:
    df = pd.DataFrame(feature)
    df_enc = pd.get_dummies(df[0])
    print(df_enc)
    df_features_final = pd.concat([df_features_final, df_enc], axis = 1, join ='inner')


print(df_features_final)

The result is the following dataframe. As you can see in the changing columntitles, after column 5 does not follow a 1:

   0  1  0  1  0  0  1  0  1  0  1  0  1
0  1  0  0  1  1  0  1  0  1  0  1  1  0
1  1  0  0  1  1  1  0  0  1  1  0  0  1
2  0  1  0  1  1  0  1  1  0  0  1  1  0
3  1  0  1  0  1  0  1  0  1  0  1  1  0
4  1  0  0  1  1  0  1  0  1  0  1  1  0


Solution

  • I don't notice the functionality you want in pandas atleast. But, in TensorFlow, we do have

    tf.one_hot(
        indices, depth, on_value=None, off_value=None, axis=None, dtype=None, name=None
    )
    

    Set depth to 2.