I wrote the following code to make a simple experiment through a Autoencoder, and I just wanna to use two losses, the first loss is the traditional MSE loss of the input and the output which is reconstructed from the latent vector of AE, and the second loss is the MSE loss between the two outputs of the symmetrical layers in encoder and the decoder, that is to say, if the AE have 5 layers, I want to add a MSE loss between the second layer and the fourth layer, for they are symmetrical.The code is here:
from time import time
import numpy as np
import random
from keras.models import Model
import keras.backend as K
from keras.engine.topology import Layer, InputSpec
from keras.layers import Dense, Input, GaussianNoise, Layer, Activation
from keras.models import Model
from keras.optimizers import SGD, Adam
from keras.utils.vis_utils import plot_model
from keras.callbacks import EarlyStopping
#build vae model
input_place = Input(shape=(128,))
e_layer1 = Dense(64,activation='relu')(input_place)
e_layer2 = Dense(32,activation='relu')(e_layer1)
hidden = Dense(16,activation='relu')(e_layer2)
d_layer1 = Dense(32,activation='relu')(hidden)
d_layer2 = Dense(64,activation='relu')(d_layer1)
output_place = Dense(128,activation='sigmoid')(d_layer2)
model = Model(inputs=input_place,outputs=output_place)
loss = K.mean(K.square(d_layer1 - e_layer2),axis = -1)
model.add_loss(loss)
model.compile(optimizer = 'adam',
loss=['mse'],
metrics=['accuracy'])
input_data = np.random.randn(400,128)
model.fit(input_data,
input_data,
batch_size = 32,
epochs=5)
but when I run this code, it occurs a error about
Epoch 1/5
32/400 [=>............................] - ETA: 12s - loss: 1.6429 - acc: 0.0000e+00Traceback (most recent call last):
File "<ipython-input-49-eac3a65824ec>", line 1, in <module>
runfile('/Users/jishilun/Desktop/keras_loss_test.py', wdir='/Users/jishilun/Desktop')
File "/anaconda3/lib/python3.7/site-packages/spyder_kernels/customize/spydercustomize.py", line 704, in runfile
execfile(filename, namespace)
File "/anaconda3/lib/python3.7/site-packages/spyder_kernels/customize/spydercustomize.py", line 108, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)
File "/Users/jishilun/Desktop/keras_loss_test.py", line 49, in <module>
epochs=5)
File "/anaconda3/lib/python3.7/site-packages/keras/engine/training.py", line 1039, in fit
validation_steps=validation_steps)
File "/anaconda3/lib/python3.7/site-packages/keras/engine/training_arrays.py", line 204, in fit_loop
callbacks.on_batch_end(batch_index, batch_logs)
File "/anaconda3/lib/python3.7/site-packages/keras/callbacks.py", line 115, in on_batch_end
callback.on_batch_end(batch, logs)
File "/anaconda3/lib/python3.7/site-packages/keras/callbacks.py", line 236, in on_batch_end
self.totals[k] += v * batch_size
ValueError: operands could not be broadcast together with shapes (32,) (16,) (32,)
and if I remove the add_loss, the code can run, so I think the two methods to add_loss in Keras/Tensorflow can't be used together simply or it may have some changes(Maybe the question occurs from the mini-batch?) Please help me! Any advice or suggestions will be welcome! Thank you very much!
The issue is not from add_loss() but from your batch_size. Your input data is (400,128), but batch_size is 32. Try to change it to a factoring number of 400, such as 40 or 20, and it will work.