Search code examples
pandascorrelationinteractionxgbclassifier

How to prevent features to interact with each other in python XGBClassifier model


I have trained this model:

model = XGBClassifier( #XGBClassifier
    objective='binary:logistic',
    base_score=0.5, 
    booster='gbtree', 
    colsample_bylevel=1,
    colsample_bynode=1, 
    colsample_bytree=1,
    enable_categorical=False, 
    gamma=2, 
    gpu_id=-1,
    importance_type=None, 
    interaction_constraints='',
    learning_rate=0.1, 
    max_delta_step=0,
    max_depth=3,
    min_child_weight=7, 
    monotone_constraints='(1,1,1,1,1)',
    n_jobs=1, 
    nthread=1, 
    num_parallel_tree=1,
    predictor='auto',
    random_state=0, 
    reg_alpha=0, 
    reg_lambda=1,
    scale_pos_weight=1, 
    silent=True, 
    subsample=0.8,
    tree_method='exact',
    validate_parameters=1, 
    pred_contribs=True,  
    verbose=True)
    
    
model.fit(X, Y)

The X dataframe has 5 predictive features. Is there a way to prevent that those 5 features interact with each other by editing the parameters in the XGBClassifier code?


Solution

  • If you change interaction_constraints=[] it will enforce that the features cannot interact.

    If you want to verify that this is the case, you could interrogate the individual tree outputs by doing something like

    trees = model.get_booster().get_dump()
    

    and then inspecting trees[0],trees[1] etc you will see that only one feature can exist per tree.