LSTM: loss value is not changing

I am working on predicting stock trend (up, or down).

Below is how I am handling my pre-processing.

index_ = len(df.columns) - 1
x = df.iloc[:,:index_]
x = x[['Relative_Volume', 'CurrentPrice', 'MarketCap']]
x = x.values.astype(float)
# x = x.reshape(len(x), 1, x.shape[1]).astype(float)
x = x.reshape(*x.shape, 1)
y = df.iloc[:,index_:].values.astype(float)

# x.shape = (44930, 3, 1)
# y.shape = (44930, 1)


x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.20, random_state=98 )

Then I am building my BILSTM model:

def build_nn():
    model = Sequential()    
    model.add(Bidirectional(LSTM(128, return_sequences=True, input_shape = (x_train.shape[0], 1) , name="one")))
    model.add(Dropout(0.20))
    model.add(Bidirectional(LSTM(128, return_sequences=True , name="two")))
    model.add(Dropout(0.20))
    model.add(Bidirectional(LSTM(64, return_sequences=False , name="three")))
    model.add(Dropout(0.20))
    model.add(Dense(1,activation='sigmoid'))
    # opt = Adam(lr=0.0001, beta_1=0.9, beta_2=0.999, decay=0.01)
    opt = SGD(lr=0.01)

    model.compile(loss='binary_crossentropy', optimizer=opt, metrics=['accuracy'])
    return model

filepath = "bilstmv1.h5"
chkp = ModelCheckpoint(monitor = 'val_accuracy', mode = 'auto', filepath=filepath, verbose = 1, save_best_only=True)


model = build_nn()
# model.summary()
model.fit(x_train, y_train,
                epochs=3,
                batch_size=256,
                validation_split=0.1, callbacks=[chkp])
model.summary()

Below is the output of the loss_value:

Epoch 1/3
127/127 [==============================] - 27s 130ms/step - loss: 0.6829 - accuracy: 0.5845 - val_loss: 0.6797 - val_accuracy: 0.5803

Epoch 00001: val_accuracy improved from -inf to 0.58025, saving model to bilstmv1.h5
Epoch 2/3
127/127 [==============================] - 14s 112ms/step - loss: 0.6788 - accuracy: 0.5851 - val_loss: 0.6798 - val_accuracy: 0.5803

Epoch 00002: val_accuracy did not improve from 0.58025
Epoch 3/3
127/127 [==============================] - 14s 112ms/step - loss: 0.6800 - accuracy: 0.5822 - val_loss: 0.6798 - val_accuracy: 0.5803

Epoch 00003: val_accuracy did not improve from 0.58025

I have tried to change the optimzer, loss_function, and other modification. As you can expect, all the predictions are same since the loss function is not being changed.

Solution

You have an issue with your input shape in your first LSTM layer. Keras inputs takes (None, Your_Shape) as its input, since your input to the model can vary. You can have 1 input, 2 inputs, or infinity inputs. The only way to represent dynamic is by using None as the first input. The quickest way to do this is to change the input to (None, *input_shape), since the * will expand your input shape.

Your build function will then become:

def build_nn():
    model = Sequential()    
    model.add(Bidirectional(LSTM(128, return_sequences=True, input_shape = (None, *x_train.shape) , name="one")))
    model.add(Dropout(0.20))
    model.add(Bidirectional(LSTM(128, return_sequences=True , name="two")))
    model.add(Dropout(0.20))
    model.add(Bidirectional(LSTM(64, return_sequences=False , name="three")))
    model.add(Dropout(0.20))
    model.add(Dense(1,activation='sigmoid'))
    # opt = Adam(lr=0.0001, beta_1=0.9, beta_2=0.999, decay=0.01)
    opt = SGD(lr=0.01)

    model.compile(loss='binary_crossentropy', optimizer=opt, metrics=['accuracy'])
    return model

Though I still advise having a look at your Optimizer as that might affect your results. You can also use -1 as an input shape which will mean auto fill, but you can only use it once.