How to set parameters for lightgbm when using customized objective function for multi-class classification?

Question

I want to test a customized objective function for lightgbm in multi-class classification. I have specified the parameter "num_class=3". However, an error: "

Number of classes must be 1 for non-multiclass training" is thrown

I am using python 3.6 and lightgbm version 0.2

# iris data
from sklearn import datasets
import lightgbm as lgb
import numpy as np

iris = datasets.load_iris()
X = iris['data']
y = iris['target']

# construct train-test
num_train = int(X.shape[0] / 3 * 2)
idx = np.random.permutation(X.shape[0])

x_train = X[idx[:num_train]]
x_test = X[idx[num_train:]]
y_train = y[idx[:num_train]]
y_test = y[idx[num_train:]]
# softmax function
def softmax(x):
    '''
    input x: an np.array of n_sample * n_class
    return : an np.array of n_sample * n_class (probabilities)
    '''
    x = np.where(x>100, 100, x)
    x = np.exp(x)
    return x / np.reshape(np.sum(x, 1), [x.shape[0], 1])
# objective function    
def objective(y_true, y_pred):
    '''
    input: 
        y_true: np.array of size (n_sample,)
        y_pred: np.array of size (n_sample, n_class)
    '''
    y_pred = softmax(y_pred) 
    temp = np.zeros_like(y_pred)
    temp[range(y_pred.shape[0]), y_true] = 1   
    gradient = y_pred - temp
    hessian = y_pred * (1 - y_pred)  
    return [gradient, hessian]
# lightgbm model
model = lgb.LGBMClassifier(n_estimators=10000,
                           num_classes = 3,
                           objective = objective,
                           nthread=4)
model.fit(x_train, y_train, 
          eval_metric = 'multi_logloss',
          eval_set = [(x_test, y_test), (x_train, y_train)],
          eval_names = ['valid', 'train'], 
          early_stopping_rounds = 200, verbose = 100)

Answer 1

Let me answer my own question.

The arguments in the objective function should be:

y_true of size [n_sample, ]
y_pred of size [n_sample * n_class, ] instead of [n_sample, n_class]

To be more specific, y_pred should be like

y_pred = [first_class, first_class,..., second_class, second_class,..., third_class, third_class,...]

Moreover, gradient and hessian should be grouped in the same way.

def objective(y_true, y_pred):
    '''
    input: 
        y_true: np.array of size [n_sample,]
        y_pred: np.array of size [n_sample * n_class, ]
    return:
        gradient and hessian should have exactly the same form of y_pred
    '''
    y_pred = np.reshape(y_pred, [num_train, 3], order = 'F')
    y_pred = softmax(y_pred)

    temp = np.zeros_like(y_pred)
    temp[range(y_pred.shape[0]), y_true] = 1

    gradient = y_pred - temp
    hessian = y_pred * (1 - y_pred)

    return [gradient.ravel(order = 'F'), hessian.ravel(order = 'F')]