I am training a keras autoencoder model with the following structure:
model = Sequential()
model.add(Dense(128, activation='relu', input_shape=(MAX_CONTEXTS, 3)))
model.add(Dense(64, activation='relu'))
model.add(Dense(32, activation='relu'))
model.add(Dense(64, activation='relu'))
model.add(Dense(128, activation='relu'))
model.add(Dense(3, activation='relu'))
model.compile(optimizer='adam', loss='mse', metrics=['accuracy'])
My data is in the shape of (number_of_samples, 430, 3) and contains values from [-1.9236537371711413, 1.9242677998256246]. This data is already normalized. I then train this model:
history = model.fit(X, X, epochs=15, batch_size=2, verbose=1, shuffle=True, validation_split=0.2)
and get an accuracy of 95.03% (suspiciously high, but my problem now is something else). Now when I predict a sample of my data, the positive values are relatively good, close to what they are in the input, but the negative values are all rounded to 0. Is this a fault of the loss function that I chose? And if so which other loss function should I choose? Or do I have to scale my data differently?
This is because you apply relu activation at the output layer.