Most of the current machine learning libraries have loss functions that comes with a weight argument, which allows us to tackle unbalanced datasets. However should this feature be actively made use of? If not, are there certain guidelines as to when we should use it(e.g. if the dataset is skewed to a certain extent). Will the model eventually learn to predict the rare cases anyway if it is complex(for the lack of a better word, I understand complexity doesn't equate to performance) enough?
I had this question because I was training a model with an unbalanced dataset(but not to the extreme), however I am adjusting the weights in the loss function somewhat arbitrarily according to the proportion of each class present in the dataset.
You can use the weighted version of loss functions if you are certain the real world data your model will need to generalize for is similarly imbalanced. If not, you are introducing man-made bias into the system.
The choice to use weight cannot be based solely on model performance during training, validation or testing, but has to be made based on close scrutiny of the data set and how it was built.
A clear example where it may help is tumor detection in CT scans, where background and foreground often have a ratio of 20:1.