What are the criteria for the weight of deeplab my custom dataset?

Question

I'm training Deeplab v3 by making custom data set in three class, including background

Then, My class is background, panda, bottle and there are 1949 pictures.

and I'm using a moblienetv2 model

and segmentation_dataset.py has been modified as follow.

_MYDATA_INFORMATION = DatasetDescriptor(
        splits_to_sizes={
            'train': 975,  # num of samples in images/training
            'trainval': 1949,
            'val': 974,  # num of samples in images/validation
        },


        num_classes=3,
        ignore_label=0,
)

train.py has been modified as follow.

flags.DEFINE_boolean('initialize_last_layer', False,
                 'Initialize the last layer.')

flags.DEFINE_boolean('last_layers_contain_logits_only', True,
                 'Only consider logits as last layers or not.')

train_utils.py has not been modified.

not_ignore_mask = tf.to_float(tf.not_equal(scaled_labels, ignore_label)) * loss_weight

I get some results, but not the perfect ones.

For example, the mask colors of panda and bottles are same or not distinct

The result that I want is panda of red and bottle of green

So, I judged that there was a problem with the weight.

Based on the other people's questions, train_utils.py was configured as follows

irgore_weight = 0
label0_weight =1
label1_weight = 10
label2_weight = 15
not_ignore_mask = 
    tf.to_float(tf.equal(scaled_labels, 0)) * label0_weight +
    tf.to_float(tf.equal(scaled_labels, 1)) * label1_weight +
    tf.to_float(tf.equal(scaled_labels, 2)) * label2_weight +
    tf.to_float(tf.equal(scaled_labels, ignore_label)) * irgore_weight 

tf.losses.softmax_cross_entropy(
    one_hot_labels,
    tf.reshape(logits, shape=[-1, num_classes]),
    weights=not_ignore_mask,
    scope=loss_scope)

I have a question here.

What are the criteria for the weight?

My data set consists of the following.

enter image description here

It's automatically generating, so I don't know exactly which one is more, but it's a similar amount.

And another thing, I'm using Pascal's color map type.

This is the first black background and the second red third green.

I want to designate pandas as red and bottles as green exactly. What should I do?

Answer 1

I think you might have mixed up your label definition. Maybe I can help you with that. Please check again your segmentation_dataset.py. Here, you define "0" as the ignored label. This means that all pixels which are labeled as "0" are excluded from the training process (more specifically, excluded in the calculation of the loss function and so have no influence in the updating of the weights). In the light of this situation it is crucial to not "ignore" the background class as it is also a class you want to predict correctly. In train_utils.py you assign a weightening factor to the ignored class which would have no effect- --> Make sure that you don't mix up your three training classes [background, panada, bottle] with the "ignored" tag.

In your case num_classes=3 should be correct as it specifies the number of labels to predict (the model automatically assumes these labels are 0, 1 and 2. If you want to ignore certain labels you have to annotate them with a fourth label class (just choose a number >2 for that) and then assign this label to ignored_label. If you don't have pixels to be ignored still set ignored_label=255 and it will not influence your training;)