Need help choosing loss function
I have used resnet50 to solve a multi-class classification problem. The model outputs probabilities for each class. Which loss function should I choose for my model?
After choosing binary cross entropy :
After choosing categorical cross entropy:
The above results are for the same model with just different loss functions.This model is supposed to classify images into 26 classes so categorical cross entropy should work. Also, in the first case accuracy is about 96% but losses are so high. Why?
edit 2:
Model architecture:
You definitely need to use categorical_crossentropy for a multi-classification problem. binary_crossentropy will reduce your problem down to a binary classification problem in a way that's unclear without further looking into it.
I would say that the reason you are seeing high accuracy in the first (and to some extent the second) case is because you are overfitting. The first dense layer you are adding contains 8 million parameters (!!! to see that do model.summary()), and you only have 70k images to train it with 8 epochs. This architectural choice is very demanding both in computing power and in data requirement. You are also using a very basic optimizer (SGD). Try to use a more powerful Adam.
Finally, I am a bit surprised at your choice to take a 'sigmoid' activation function in the output layer. Why not a more classic 'softmax'?
- How can I tell when the model is overfitting?
- StratifiedKFold vs KFold in scikit-learn
- Training a Keras model to identify leap years
- Ideas for Extracting Blade Tip Coordinates from masked Wind Turbine Image
- Macro VS Micro VS Weighted VS Samples F1 Score
- Doing PyWavelets calculation on GPU
- Training loss increases instead of decrease with epochs
- How to save a Dataset in multiple shards using `tf.data.Dataset.save`
- why explain logit as 'unscaled log probabililty' in sotfmax_cross_entropy_with_logits?
- What is the loss function used in Trainer from the Transformers library of Hugging Face?
- Using features extracted using a pretrained CNN as new features for an CNN/NN
- InvalidArgumentError: No DNN in stream executor while training a TensorFlow RetinaNet model on Google Colab
- ALS (Alternating Least Square) algorithm in multiple rankings for a user
- How does one set the pad token correctly (not to eos) during fine-tuning to avoid model not predicting EOS?
- How to create image of confusion matrix in Python
- Cross-validation with nb method
- GPU utilization almost always 0 during training Hugging Face Transformer
- The “Forward/Backward Passage Size” is too large for the pytorch model (Yolov3)
- How many images(minimum) should be there in each classes for training YOLO?
- Why do neural networks work so well?
- Will larger batch size make computation time less in machine learning?
- Why KL divergence is negative in Pytorch?
- Creating a voice identification system using machine learning
- Forward pass with all samples
- override pytorch Dataset efficiently
- fit method in sklearn
- Calibrating Probabilities in lightgbm or XGBoost
- Implementation of F1-score, IOU and Dice Score
- Is it ok to have the training history very similar to the validation history?
- How to understand Shapley value for binary classification problem?