Search code examples
pythonneural-networkkerassequential

keras sequential().predict(x_test) only returns 1 column for two classes

I'm having problem with keras sequential().predict(x_test).

Btw getting the same output using sequential().predict_proba(x_test) as I found that these two are indifferent in sequential now.

My data has two classes: 0 or 1, I believe predict(x_test) should give two columns, where the first column is the prob for getting 0 and the second is prob of getting 1. However I only have one column with this.

    In [85]:y_train.value_counts()
    Out[85]: 
    0    616751
    1     11140
    Name: _merge, dtype: int64

There should be no problem with my data as I used the same x_train, y_train, x_test, y_test for both LogisticRegression model and neural network model, it works perfect in LogisticRegression.

In [87]:y_pred_LR
Out[87]: 
array([[  9.96117151e-01,   3.88284921e-03],
       [  9.99767583e-01,   2.32417329e-04],
       [  9.87375774e-01,   1.26242258e-02],
       ..., 
       [  9.72159138e-01,   2.78408623e-02],
       [  9.97232916e-01,   2.76708432e-03],
       [  9.98146985e-01,   1.85301489e-03]])

but I only get 1 column in neural network model.

So I guess there is some problem with the NN model setting up? Here is my codes

NN = Sequential()
NN.add(Dense(40, input_dim = 65, kernel_initializer = 'uniform', activation = 'relu'))
NN.add(Dense(20, kernel_initializer = 'uniform', activation = 'relu'))
NN.add(Dense(1, kernel_initializer = 'uniform', activation = 'sigmoid'))
NN.compile(optimizer = 'adam', loss = 'binary_crossentropy', metrics = ['accuracy'])

NN.fit(x_train, y_train, batch_size = 50, epochs=5)
y_pred_NN = NN.predict(x_test)
print(y_pred_NN)

    In [86]: print(y_pred_NN)
    [[ 0.00157279]
     [ 0.0010451 ]
     [ 0.03178826]
     ..., 
     [ 0.01030775]
     [ 0.00584918]
     [ 0.00186538]]

Actually it looks like it's the prob of getting 1? Any help is appreciated!

Btw the shapes of my predictions in both models are as follows

In [91]:y_pred_LR.shape
Out[91]: (300000, 2)

In [90]:y_pred_NN.shape
Out[90]: (300000, 1)
Solution

  • If you want to output two probabilities, you will have to replace your y_train with to_categorical(y_train) and then adjust the network accordingly:

    from keras.utils import to_categorical

NN = Sequential()
NN.add(Dense(40, input_dim = 65, kernel_initializer = 'uniform', activation = 'relu'))
NN.add(Dense(20, kernel_initializer = 'uniform', activation = 'relu'))
NN.add(Dense(2, activation='sigmoid'))
NN.compile(optimizer = 'adam', loss = 'binary_crossentropy', metrics = ['accuracy'])

NN.fit(x_train, to_categorical(y_train), batch_size = 50, epochs=5)

    Consult here: https://keras.io/utils/#to_categorical