I have to create model which returns 3 numbers - the angles by which the 3d model is rotated. All models I use are rotated in the same way in the beginning. For this I rotate every model for a random degree from -15 to 15 and get this five times for each model (the color represents distance from "camera"). Each of these images has size [150, 150] and contain numbers from 0 to 255:
Top: Click
Side: Click
Front: Click
Then I've created a model:
My training dataset is 30 models and for each I create 5 different rotations, so there's 150 samples. I used 105 for training and 45 for validation
Then, when I've tried to fit it for 100 epochs, batch size = 40, Adam, learning rate = 0.1, error - mean absolute error. I use my custom metric, it's just sum of abs(angle_true - angle_pred). It showed me this and something extremely similar during all training:
Epoch 95/100 3/3 [==============================] - 4s 1s/step - loss: 6.8082 - metr: 25.7488 - val_loss: 7.8395 - val_metr: 16.2496
Epoch 96/100 3/3 [==============================] - 4s 1s/step - loss: 6.8203 - metr: 28.0160 - val_loss: 7.8005 - val_metr: 16.1032
Epoch 97/100 3/3 [==============================] - 4s 1s/step - loss: 6.7898 - metr: 21.8611 - val_loss: 7.8166 - val_metr: 15.6580
Epoch 98/100 3/3 [==============================] - 4s 1s/step - loss: 6.7806 - metr: 18.9963 - val_loss: 7.8766 - val_metr: 15.1825
Epoch 99/100 3/3 [==============================] - 4s 1s/step - loss: 6.7973 - metr: 21.3751 - val_loss: 7.8581 - val_metr: 14.8261
Epoch 100/100 3/3 [==============================] - 4s 1s/step - loss: 6.7750 - metr: 19.1987 - val_loss: 7.8100 - val_metr: 14.2220
There's what model actually predicts (five it's predictions vs five true angles):
[[-2.9442613 -0.54196733 2.4832273 ]
[-4.1728406 -0.60556126 1.6946273 ]
[-4.727777 -1.2003385 0.9080634 ]
[-3.125803 0.2552796 1.7383668 ]
[-5.6989026 0.5544905 1.539639 ]
[-3.6019537 0.29354924 1.2323818 ]]
[[ -4. -8. 2.]
[-12. -7. 4.]
[-13. -4. -3.]
[ -7. 2. -4.]
[-13. 8. -12.]
[-15. -11. -11.]]
I recreated it some times and, as you can see, results were always close to one number and didn't varied. I don't know why they can't reach ewen 10.
I've tried to add SeparableConv2D after every Conv2D layer, to change loss function and to change learning rate to 0.01 and 0.001 but nothing changed. Why the model doesn't fit?
Dataset is too small for regression task. Try to add more samples and use data augmentation (e.g. horizontal flipping, shifting, zooming, color jittering).