I'm trying to implement Huber loss to make customization for MAPE loss in lightgbm. Below is my code. However, when I try to run it I get zeros for all predictions. What is wrong with the code? It seems that some scalling could a bit help with learning, but I don't see any guidelines in the internet on how it should be applied inside customized loss. Could you please help me with that?
def my_loss(preds, dtrain):
y_true = dtrain.get_label()
d = (preds - y_true)
h = 1 #h is delta in the graphic
scale = 1 + (d / h) ** 2
scale_sqrt = np.sqrt(scale)
grad = d / scale_sqrt
hess = 1 / scale / scale_sqrt
hess = np.ones(len(preds))
return grad, hess
metrics = []
for i in my_cv:
X_train = X.loc[i[0],:]
y_train = y.loc[i[0]]
X_test = X.loc[i[1],:]
y_test = y.loc[i[1]]
dtrain = xgb.Dataset(X_train, label=y_train, free_raw_data =False)
params = {'max_depth': 10, 'learning_rate':0.05,'objective':None,
'num_leaves':150, 'min_child_samples':5, 'nround':100,
'monotone_constraints':lst_mon}
mm = xgb.train(params, dtrain, fobj = my_loss)
y_pred = mm.predict(X_train)
the correct function:
def my_loss(preds, dtrain):
y_true = dtrain.get_label()
d = (preds - y_true)
h = 1 #h is delta in the graphic
scale = 1 + (d / h) ** 2
scale_sqrt = np.sqrt(scale)
grad = d / scale_sqrt
hess = 1 / scale / scale_sqrt
return grad, hess
removed hess = np.ones(len(preds))