I have a question about this algorithm:
Partition training data in “grow” and “validation” sets.
Build a complete tree from the “grow” data.
Until accuracy on validation set decreases do:
For each non-leaf node, n, in the tree do:
Temporarily prune the subtree below n and replace it with a leaf labeled with
the current majority class at that node.
Measure and record the accuracy of the pruned tree on the validation set.
Permanently prune the node that results in the greatest increase in accuracy on the
validation set.
I don't understand the part "Permanently prune the node that results in the greatest increase in accuracy on the validation set." We are supposed to keep the nodes that increase the accuracy and prune those that increase the error rate. Am I wrong?
I don't even know which realm this algorithm applies to, but it's my understanding that the nodes that increase accuracy are the ones that are not pruned, so there is no contradiction in the phrase you quote. Maybe it could be rephrased
permanently prune the node that, when pruned, causes the greatest increase in accuracy on the validation set
to make it clearer.