loss value and val_loss value not decreasing in RNN algorithm for continuous data prediction

Question

I am building a model for house price prediction using RNN and below is the code. The dataset has no null values and is completely cleaned, still I am getting constant and high loss and val_loss values. how can I make these values decrease values?

A = dataset.drop(['price'],axis="columns")
B = dataset['price']

from sklearn import preprocessing
min_max_scaler = preprocessing.MinMaxScaler()
A_scale = min_max_scaler.fit_transform(A)

from sklearn.model_selection import train_test_split
A_train, A_test, B_train, B_test = train_test_split(A_scale, B, test_size=0.3)
a_val, a_test, b_val, b_test = train_test_split(A_test, B_test, test_size=0.5)

from keras.models import Sequential
from keras.layers import Dense,LSTM,Dropout
regressor = Sequential()

model = Sequential([
Dense(32, activation='relu', input_shape=(10,)),
Dense(32, activation='relu'),
Dense(1, activation='sigmoid'),
])
model.compile(optimizer='adam',loss='mse',metrics=['mae'])

hist = model.fit(A_train, B_train, batch_size=32, epochs=4, validation_data=(a_val, b_val))

output:

Epoch 1/20
292/292 [==============================] - 0s 1ms/step - loss: 36314.9180 - mae: 111.9050 - val_loss: 23161.0312 - val_mae: 106.9015
Epoch 2/20
292/292 [==============================] - 0s 646us/step - loss: 36295.7930 - mae: 111.8202 - val_loss: 23160.9219 - val_mae: 106.9010
Epoch 3/20
292/292 [==============================] - 0s 715us/step - loss: 36295.7383 - mae: 111.8199 - val_loss: 23160.9121 - val_mae: 106.9009
Epoch 4/20
292/292 [==============================] - 0s 716us/step - loss: 36295.7422 - mae: 111.8199 - val_loss: 23160.9082 - val_mae: 106.9009

Answer 1

It could mean many things, but three things come to mind:

1. Tweaking the learning rate in your model hyperparameters is very important. This will give you some context on what Learning rate is :)
2. Adding some more epochs to the model will help it converge to a local minima.
3. If you are doing regression, use the Linear activation function.

To implement it, try with the following:

from keras.optimizers import Adam
from keras.models import Sequential
from keras.layers import Dense

LR=0.001
EPOCHS=100
BATCH_SIZE=32

opt = Adam(lr=LR, decay=LR/EPOCHS)

model = Sequential([
Dense(32, activation='relu', input_shape=(10,)),
Dense(32, activation='relu'),
Dense(1, activation='linear'),
])
model.compile(optimizer=opt, loss='mse', metrics=['mae'])

hist = model.fit(A_train, B_train, batch_size=BATCH_SIZE, epochs=EPOCHS, validation_data=(a_val, b_val))

I encourage you to experiment, trial and error, read about all the hyperparameters and their effects, and try with different combinations on each layer of the neural network.