I am trying to change a neural network that classifies pointclouds into 40 different classes, to a regression networks that predicts a specific property of them in three points(x,y,z). for the loss function I have replaced this loss value:
loss = tf.nn.sparse_softmax_cross_entropy_with_logits(logits=pred, labels=label)
classify_loss = tf.reduce_mean(loss)
tf.summary.scalar('classify loss', classify_loss)
with this one:
mse = tf.keras.losses.MeanSquaredError()
regression_loss=mse(pred, label)
regression_loss = tf.cast(regression_loss, tf.float32)
tf.summary.scalar('regression loss', regression_loss)
I have only changed the last layer output from 40 to 3 and some minor stuff for the network to actually start the training.
I used tf.cast because there is another loss computed in the middle of network multiplied by 0.001 and added to this loss to give the mean loss and before that, it kept raising the error that x(regression_loss) is int and y(middle_loss) is float and they can't be added which was also weird. The values for 'labels' are small around zero but the loss stays at zero almost all the time except for random times. I would be thankful if anyone could help me find the problem or replace the loss function maybe.
The log looks like this:
**** EPOCH 000 ****
----0-----
mean loss: 0.000000
----1-----
mean loss: 0.000000
----2-----
mean loss: 0.000000
----3-----
mean loss: 0.000000
----4-----
mean loss: 0.000000
----0-----
----1-----
eval mean loss: 0.000000
Model saved in file: log/model.ckpt
**** EPOCH 001 ****
----0-----
mean loss: 0.000000
----1-----
mean loss: 0.000000
----2-----
mean loss: 0.000000
----3-----
mean loss: 0.000000
----4-----
mean loss: 0.000000
----0-----
----1-----
eval mean loss: 0.000000
**** EPOCH 002 ****
----0-----
mean loss: 0.000000
----1-----
mean loss: 0.019608
----2-----
mean loss: 0.000000
----3-----
mean loss: 0.015625
----4-----
mean loss: 0.000000
----0-----
----1-----
eval mean loss: 0.000000
**** EPOCH 003 ****
----0-----
mean loss: 0.000000
----1-----
mean loss: 0.000000
----2-----
mean loss: 0.000000
----3-----
mean loss: 0.000000
----4-----
mean loss: 0.000000
----0-----
----1-----
eval mean loss: 0.000000
**** EPOCH 004 ****
----0-----
mean loss: 0.000000
----1-----
mean loss: 0.000000
----2-----
mean loss: 0.000000
----3-----
mean loss: 0.000000
----4-----
mean loss: 0.000000
----0-----
----1-----
eval mean loss: 0.000000
**** EPOCH 005 ****
----0-----
mean loss: 0.019608
----1-----
mean loss: 0.000000
----2-----
mean loss: 0.000000
----3-----
mean loss: 0.000000
----4-----
mean loss: 0.000000
----0-----
----1-----
eval mean loss: 0.000000
**** EPOCH 006 ****
----0-----
mean loss: 0.000000
----1-----
mean loss: 0.000000
----2-----
mean loss: 0.000000
----3-----
mean loss: 0.000000
----4-----
mean loss: 0.000000
----0-----
----1-----
eval mean loss: 0.000000
**** EPOCH 007 ****
----0-----
mean loss: 0.000000
----1-----
mean loss: 0.000000
----2-----
mean loss: 0.015625
----3-----
mean loss: 0.015625
----4-----
mean loss: 0.000000
----0-----
----1-----
eval mean loss: 0.000000
**** EPOCH 008 ****
----0-----
mean loss: 0.000000
----1-----
mean loss: 0.000000
----2-----
mean loss: 0.000000
----3-----
mean loss: 0.000000
----4-----
mean loss: 0.000000
----0-----
----1-----
eval mean loss: 0.000000
**** EPOCH 009 ****
----0-----
mean loss: 0.015625
----1-----
mean loss: 0.000000
----2-----
mean loss: 0.000000
----3-----
mean loss: 0.000000
----4-----
mean loss: 0.000000
----0-----
----1-----
eval mean loss: 0.000000
**** EPOCH 010 ****
----0-----
mean loss: 0.000000
----1-----
mean loss: 0.015625
----2-----
mean loss: 0.000000
----3-----
mean loss: 0.000000
----4-----
mean loss: 0.000000
----0-----
----1-----
eval mean loss: 0.000000
Model saved in file: log/model.ckpt
**** EPOCH 011 ****
----0-----
mean loss: 0.000000
----1-----
mean loss: 0.000000
----2-----
mean loss: 0.000000
----3-----
mean loss: 0.000000
----4-----
mean loss: 0.000000
----0-----
----1-----
eval mean loss: 0.000000
**** EPOCH 012 ****
----0-----
mean loss: 0.000000
----1-----
mean loss: 0.000000
----2-----
mean loss: 0.000000
----3-----
mean loss: 0.000000
----4-----
mean loss: 0.000000
----0-----
----1-----
eval mean loss: 0.000000
**** EPOCH 013 ****
----0-----
mean loss: 0.000000
----1-----
mean loss: 0.000000
----2-----
mean loss: 0.000000
----3-----
mean loss: 0.000000
----4-----
mean loss: 0.000000
----0-----
----1-----
eval mean loss: 0.000000
**** EPOCH 014 ****
----0-----
mean loss: 0.000000
----1-----
mean loss: 0.015625
----2-----
I have thought about multiplying the regression_loss by 100 but I don't believe that would make much difference.
I figured out the reason for this the same day:) in case anyone comes across this kind of problem this is what I did wrong:
for the labels placeholder, I forgot to change the tf.int32 to tf.float32 so the labels were all transformed to int32 and it would constantly calculate zero for the loss!
labels_pl = tf.placeholder(tf.float32, shape=(batch_size,3))