Minimize the small loss to zero loss in transfer learning
I am asking this question because I noticed in competitions people tend to minimize the loss to 0. I have an "image binary classification " problem and I already achieved the binary_crossentropy_loss to 0.003 with a "train from scratch" transfer learning model. How can I further reduce it to 0? Should I fine-tune the model again or should I go back to do image feature engineering?
Additionally, according to the picture here, I suppose I encountered "vanished gradient" instead of "overfitting". If so, what should I do on the next step?
Thank you!
Since you are trying to perform image binary classification, if you can minimize both your training and validation loss to 0, that basically means your network is 'perfectly' trained to recognize all the validation images by using just the training images. When this happens, I think it's better for you to get 'harder' data for your network to learn.
From your image, I think you should continue training your model for more epochs, since val_loss does not seem to converge yet; as a result, there are no indications of 'overfitting'.
Regarding 'vanished gradient', it's not possible to tell from your picture since the common sign of vanishing gradients is weights dying down to 0. To check for this problem, I think you should keep track of the weights distribution of your model in addition to the losses.
- The “Forward/Backward Passage Size” is too large for the pytorch model (Yolov3)
- How do I use distributed DNN training in TensorFlow?
- Neural network learning to sum two numbers
- Implementation of F1-score, IOU and Dice Score
- use matplotlib_inline and torch、d2l show error :NotImplementedError: Implement enable_gui in a subclass
- how to implement custom metric in keras?
- torchrl: Using SyncDataCollector with a custom pytorch dqn
- Does peft train newly initialized weights?
- Do I have to write custom AutoModel transformers class in case "TypeError: NVEmbedModel.forward() got an unexpected keyword argument 'inputs_embeds'"
- Why RAG is slower than LLM?
- "RuntimeError: Numpy is not available" when using inverse_transform
- Pytorch RuntimeError: "host_softmax" not implemented for 'torch.cuda.LongTensor'
- AMD ROCm with Pytorch on Navi10 (RX 5700 / RX 5700 XT)
- Can we use multiple loss functions in same layer?
- How do I update pixelClassificationLayer() to a custom loss function?
- Neuralnet RMSE is 10x bigger than linear model's RMSE on test data set
- Back Propagation in Convolutional Neural Networks and how to update filters
- Face alignment megaface
- autoencoder.fit() raises 'KeyError: 'Exception encountered when calling Functional.call()'
- When to use numpy.random.randn(...) and when numpy.random.rand(...)?
- What is freezing/unfreezing a layer in neural networks?
- How can I use a pre-trained neural network with grayscale images?
- PyTorch RuntimeError: device >= 0 && device < num_gpus INTERNAL ASSERT FAILED
- How do I initialize weights in PyTorch?
- Does one convolutional filter always have different coefficients for each of the channels of the previous layer?
- Obtain the output of intermediate layer (Functional API) and use it in SubClassed API
- Optuna Hyperband Algorithm Not Following Expected Model Training Scheme
- Broadcasting multiple versions of X_data that pair with the same y_data
- How to make TensorFlow use 100% of GPU?
- Contrastive Loss from Scratch