Search code examples
rr-caretadaboost

R: How to read ada's AdaBoost tree rules in 'if-else' conditions?


Does anyone know how to transform the AdaBoost trees (results in R) into if-else conditions?

I have used the caret package in R, along with the train function and method="ada" to obtain some predictions on my dataset.

Afterwards I used the function listTreesAda in the rattle package to see their structure (sample below):

n= 45211
node), split, n, loss, yval, (yprob)
* denotes terminal node
1) root 45211 5280 no (0.88321426 0.11678574)
2) duration< 551.5 40743 3245 no (0.92035442 0.07964558)
4) poutcome=failure,other,unknown 39451 2431 no (0.93837926 0.06162074) *
5) poutcome=success 1292 478 yes (0.36996904 0.63003096)
10) duration=132.5 1066 300 yes (0.28142589 0.71857411) *
3) duration>=551.5 4468 2035 no (0.54453894 0.45546106)
6) duration=835.5 1688 697 yes (0.41291469 0.58708531) *

How do you read it in if-else conditions? If you wanted to calculate the final function of the AdaBoost model yourself, how would you do it? Having a weight per tree {w(1), w(2), ..., w(n)}, what would you multiply this weight with exactly? The rule corresponding to the final leaves where the response variable is YES? ... where the response variable is NO? ... with each if-else rule of the tree?

As you can see I am a little bit lost.

(This post is related to: https://stats.stackexchange.com/questions/133088/use-adaboost-results-in-r-to-make-future-predictions-by-hand and https://stackoverflow.com/questions/27857619/r-error-using-drawtreesada-to-draw-an-ada-tree)


Solution

  • After much struggle I've got it. At least reading the if-else conditions... It goes like this:

                                              ____N=45211_____
                                             |duration < 551.5|
                                             |________________|
                                                     /\     
                   _______(rule=TRUE, Y=no)_________/  \_______(rule=FALSE, Y=no)________ 
                  |                                                                      | 
    ______________|_________________                                       ______________|_____________         
    |poutcome=failure,other,unknown|                                      |      duration=835.5        |
    |______________________________|                                      |____________________________|
                  /\                                                                    /\ 
                 /  \                                                                  /  \
    ____________/_   \__(rule=F,Y=yes)__                                 (rule=T, Y=yes)  (rule=F, Y=...)
    |(rule=T,Y=no)|  | duration=132.5  |                                      N=1688        N=(4468-1688)
    |_____________|  |_________________|
       N=39451               /\
                            /  \
              (rule=T, Y=yes)  (rule=F, Y=... partial output)
                  N=1066            N=(1292-1066)=226