Search code examples
pythonkeraslogistic-regression

logistic regression classifier by keras

I am trying to write a logistic regression model by keras.But I find out some problems:

The data I use is from Coursera Machine learning course(taught by Andrew NG) ex2.

and my code is below:

def model():
input_layer = Input(shape=(2,))
dense1 = Dense(1,activation='sigmoid',kernel_initializer=RandomNormal(seed=42),bias_initializer='zeros')(input_layer)
model = Model(input_layer,dense1)

return model
model.compile(loss=keras.losses.binary_crossentropy,optimizer=Adam(),metrics=['accuracy']) 
model.fit(x=X,y=y,batch_size=10,epochs=10,verbose=1)

Here is the result I get: 100/100 [==============================] - 0s 120us/step - loss: 0.6335 - acc: 0.6000 Epoch 10/10 100/100 [==============================] - 0s 130us/step - loss: 0.6326 - acc: 0.6000

But if I use matlab fminunc function to find the result, I get: Train Accuracy: 89.000000

Why the result can be different?

Thanks

Solution

  • First make sure you have normalized the input data. Try normalizing it like this:

    X = X.astype('float32')
X_mean = X.mean(axis=0)
X -= X_mean
X_std = X.std(axis=0)
X /= X_std + 1e-8

    Further, increasing the learning rate might also help in this case as well: optimizer=Adam(lr=1e-2), it helps the training to converge to a solution in fewer epochs.