Ordinal Encoding or One-Hot-Encoding

Question

IF we are not sure about the nature of categorical features like whether they are nominal or ordinal, which encoding should we use? Ordinal-Encoding or One-Hot-Encoding? Is there a clearly defined rule on this topic?

I see a lot of people using Ordinal-Encoding on Categorical Data that doesn't have a Direction. Suppose a frequency table:

some_data[some_col].value_counts()
[OUTPUT]
color_white    11413
color_green     4544
color_black     1419
color_orang        3
Name: shirt_colors, dtype: int64

There are a lots of guys who are preferring to do Ordinal-Encoding on this column. And I am hell-bent to go with One-Hot-Encoding. My view on this is that doing Ordinal Encoding will allot these colors' some ordered numbers which I'd imply a ranking. And there is no ranking in the first place. In other words, my model should not be thinking of color_white to be 4 and color_orang to be 0 or 1 or 2. Keep in mind that there is no hint of any ranking or order in the Data Description as well.

I have the following understanding of this topic:

Numbers that neither have a direction nor magnitude are Nominal Variables. For example, fruit_list =['apple', 'orange', banana']. Unless there is a specific context, this set would be called to be a nominal one. And for such variables, we should perform either get_dummies or one-hot-encoding

Whereas the Ordinal Variables have a direction. For example, shirt_sizes_list = [large, medium, small]. These variables are called Ordinal Variables. If the same fruit list has a context behind it, like price or nutritional value i-e, that could give the fruits in the fruit_list some ranking or order, we'd call it an Ordinal Variable. And for Ordinal Variables, we perform Ordinal-Encoding

Is my understanding correct? Kindly provide your feedback This topic has turned into a nightmare Thank you!

Answer 1

Just one thing to consider for choosing OrdinalEncoder or OneHotEncoder is that does the order of data matter?

Most ML algorithms will assume that two nearby values are more similar than two distant values. This may be fine in some cases e.g., for ordered categories such as:

• quality = ["bad", "average", "good", "excellent"] or
• shirt_size = ["large", "medium", "small"]

but it is obviously not the case for the:

• color = ["white","orange","black","green"]

column (except for the cases where you need to consider a spectrum, say from white to black. Note that in this case, white category should be encoded as 0 and black should be encoded as the highest number in your categories), or if you have some cases for example, say, categories 0 and 4 may be more similar than categories 0 and 1. To fix this issue, a common solution is to create one binary attribute per category (One-Hot encoding)