Search code examples
classificationdecision-treeentropy

Entropy and Decision Trees


Suppose I have a table of customer information with attributes such as customer ID, name, date of birth, nationality, income, etc.

Each customer in the table has a unique customer ID. I know that the Gini coefficient for each Customer ID value is zero yielding an overall Gini for Customer ID to be zero.

Can I also assume that the entropy of Customer ID is also zero? Why or why not?


Solution

  • Yes. Both Gini and entropy quantify impurity, and a value of 0 indicates perfect purity. Thus, if Gini equals 0, so does entropy.