I'm trying to get the best classifier for a data set on Weka and I'm studying different types of maximum depth for the Random Tree algorithm. But I don't understand the results I get: with a maximum-Depth between 3 and 10 I get a far better accuracy rate than with a maximum-Depth>10. Anyone can help me to figure out why? Deeper trees shouldn't give better accuracy ?
Deeper tree gives better accuracy on the training set, not on the testing one. Deep tree lets your model overfit to your data better, create more closely fitted decision boundary, which often does not correspond to actual boundary between classes.