I'm trying to train AlexNet with multi class lmdb data. The input layers are:
layer {
name: "data"
type: "Data"
top: "data"
data_param {
source: "/home/gal/caffe-1.0/models/mixDCNN/hdf5/TrainImage"
backend: LMDB
batch_size: 64
}
transform_param {
crop_size: 227
mean_file: "/home/gal/caffe-1.0/models/mixDCNN/hdf5/TrainImage/TrainImage.binaryproto"
mirror: true
}
include: { phase: TRAIN }
}
layer {
name: "labels"
type: "Data"
top: "labels"
include {
phase: TRAIN
}
data_param {
source: "/home/gal/caffe-1.0/models/mixDCNN/hdf5/TrainLabel"
batch_size: 64
backend: LMDB
}
}
layer {
name: "data"
type: "Data"
top: "data"
data_param {
source: "/home/gal/caffe-1.0/models/mixDCNN/hdf5/ValImage"
backend: LMDB
batch_size: 64
}
transform_param {
crop_size: 227
mean_file: "/home/gal/caffe-1.0/models/mixDCNN/hdf5/ValImage/ValImage.binaryproto"
mirror: false
}
include: { phase: TEST }
}
layer {
name: "labels"
type: "Data"
top: "labels"
include {
phase: TEST
}
data_param {
source: "/home/gal/caffe-1.0/models/mixDCNN/hdf5/ValLabel"
batch_size: 64
backend: LMDB
}
}
layer {
name: "slice"
type: "Slice"
bottom: "labels"
top: "class"
top: "subclass"
top: "color"
slice_param {
axis: 1
slice_point: 1
slice_point: 2
}
}
And the outputs layers are:
layer {
name: "fc8_1_class"
type: "InnerProduct"
bottom: "fc7"
top: "fc8_1_class"
param {
lr_mult: 5
decay_mult: 1
}
param {
lr_mult: 10
decay_mult: 0
}
inner_product_param {
num_output: 2
weight_filler {
type: "gaussian"
std: 0.01
}
bias_filler {
type: "constant"
value: 0
}
}
}
layer {
name: "fc8_2_subclass"
type: "InnerProduct"
bottom: "fc7"
top: "fc8_2_subclass"
param {
lr_mult: 5
decay_mult: 1
}
param {
lr_mult: 10
decay_mult: 0
}
inner_product_param {
num_output: 15
weight_filler {
type: "gaussian"
std: 0.01
}
bias_filler {
type: "constant"
value: 0
}
}
}
layer {
name: "fc8_3_color"
type: "InnerProduct"
bottom: "fc7"
top: "fc8_3_color"
param {
lr_mult: 5
decay_mult: 1
}
param {
lr_mult: 10
decay_mult: 0
}
inner_product_param {
num_output: 8
weight_filler {
type: "gaussian"
std: 0.01
}
bias_filler {
type: "constant"
value: 0
}
}
}
layer {
name: "accuracy_class"
type: "Accuracy"
bottom: "fc8_1_class"
bottom: "class"
top: "accuracy_class"
include: { phase: TEST }
}
layer {
name: "loss_class"
type: "SoftmaxWithLoss"
bottom: "fc8_1_class"
bottom: "class"
top: "loss_class"
loss_weight: 0.5
}
layer {
name: "accuracy_subclass"
type: "Accuracy"
bottom: "fc8_2_subclass"
bottom: "subclass"
top: "accuracy_subclass"
include: { phase: TEST }
}
layer {
name: "loss_subclass"
type: "SoftmaxWithLoss"
bottom: "fc8_2_subclass"
bottom: "subclass"
top: "loss_subclass"
loss_weight: 0.5
}
layer {
name: "accuracy_color"
type: "Accuracy"
bottom: "fc8_3_color"
bottom: "color"
top: "accuracy_color"
include: { phase: TEST }
}
layer {
name: "loss_color"
type: "SoftmaxWithLoss"
bottom: "fc8_3_color"
bottom: "color"
top: "loss_color"
loss_weight: 0.5
}
"Class" is a dominant class with 2 labels which gives good accuracy, on the other hand "SubClass" has 15 labels and gives poor results, "Color" keeps giving me accuracy=0 no matter what I tryed. This is part of train output:
I1123 10:47:52.258751 3392 solver.cpp:273] Learning Rate Policy: step
I1123 10:47:52.422647 3392 solver.cpp:218] Iteration 0 (7.96684e-08 iter/s, 0.163777s/100 iters), loss = 3.50787
I1123 10:47:52.422716 3392 solver.cpp:237] Train net output #0: loss_class = 1.83086 (* 0.5 = 0.915429 loss)
I1123 10:47:52.422742 3392 solver.cpp:237] Train net output #1: loss_color = 2.39675 (* 0.5 = 1.19837 loss)
I1123 10:47:52.422768 3392 solver.cpp:237] Train net output #2: loss_subclass = 2.78814 (* 0.5 = 1.39407 loss)
I1123 10:47:52.422803 3392 sgd_solver.cpp:105] Iteration 0, lr = 0.0001
I1123 10:48:01.282282 3392 solver.cpp:330] Iteration 100, Testing net (#0)
I1123 10:48:01.554111 3392 blocking_queue.cpp:49] Waiting for data
I1123 10:48:03.717224 3406 data_layer.cpp:73] Restarting data prefetching from start.
I1123 10:48:03.884392 3405 data_layer.cpp:73] Restarting data prefetching from start.
I1123 10:48:04.193727 3392 solver.cpp:397] Test net output #0: accuracy_class = 0.958594
I1123 10:48:04.193784 3392 solver.cpp:397] Test net output #1: accuracy_color = 0
I1123 10:48:04.193802 3392 solver.cpp:397] Test net output #2: accuracy_subclass = 0.495313
I1123 10:48:04.193841 3392 solver.cpp:397] Test net output #3: loss_class = 0.143002 (* 0.5 = 0.0715012 loss)
I1123 10:48:04.193871 3392 solver.cpp:397] Test net output #4: loss_color = 1.53319 (* 0.5 = 0.766597 loss)
I1123 10:48:04.193894 3392 solver.cpp:397] Test net output #5: loss_subclass = 1.40099 (* 0.5 = 0.700494 loss)
I1123 10:48:04.290772 3392 solver.cpp:218] Iteration 100 (8.42592 iter/s, 11.8681s/100 iters), loss = 1.50249
I1123 10:48:04.302697 3392 solver.cpp:237] Train net output #0: loss_class = 0.145322 (* 0.5 = 0.0726612 loss)
I1123 10:48:04.302740 3392 solver.cpp:237] Train net output #1: loss_color = 1.44309 (* 0.5 = 0.721547 loss)
I1123 10:48:04.302767 3392 solver.cpp:237] Train net output #2: loss_subclass = 1.41657 (* 0.5 = 0.708286 loss)
I1123 10:48:04.302788 3392 sgd_solver.cpp:105] Iteration 100, lr = 0.0001
I1123 10:48:08.030251 3404 data_layer.cpp:73] Restarting data prefetching from start.
I1123 10:48:08.077759 3403 data_layer.cpp:73] Restarting data prefetching from start.
I1123 10:48:13.195049 3392 solver.cpp:330] Iteration 200, Testing net (#0)
I1123 10:48:15.287199 3406 data_layer.cpp:73] Restarting data prefetching from start.
I1123 10:48:15.466104 3405 data_layer.cpp:73] Restarting data prefetching from start.
I1123 10:48:16.056556 3392 solver.cpp:397] Test net output #0: accuracy_class = 0.958594
I1123 10:48:16.056615 3392 solver.cpp:397] Test net output #1: accuracy_color = 0
I1123 10:48:16.056633 3392 solver.cpp:397] Test net output #2: accuracy_subclass = 0.520703
I1123 10:48:16.056658 3392 solver.cpp:397] Test net output #3: loss_class = 0.11685 (* 0.5 = 0.0584251 loss)
I1123 10:48:16.056679 3392 solver.cpp:397] Test net output #4: loss_color = 1.53322 (* 0.5 = 0.766609 loss)
I1123 10:48:16.056707 3392 solver.cpp:397] Test net output #5: loss_subclass = 1.42879 (* 0.5 = 0.714395 loss)
I1123 10:48:16.151207 3392 solver.cpp:218] Iteration 200 (8.43993 iter/s, 11.8484s/100 iters), loss = 1.74936
I1123 10:48:16.163187 3392 solver.cpp:237] Train net output #0: loss_class = 0.253774 (* 0.5 = 0.126887 loss)
I1123 10:48:16.163224 3392 solver.cpp:237] Train net output #1: loss_color = 1.69651 (* 0.5 = 0.848255 loss)
I1123 10:48:16.163246 3392 solver.cpp:237] Train net output #2: loss_subclass = 1.54843 (* 0.5 = 0.774216 loss)
I1123 10:48:16.163265 3392 sgd_solver.cpp:105] Iteration 200, lr = 0.0001
The data contains around 9000 images for train and another 2000 for validation, labels are in this form (starts from 0):
/media/gal/USB/MAFAT/crops/train/img/15036.png 0 4 18
/media/gal/USB/MAFAT/crops/train/img/31658.png 1 7 17
/media/gal/USB/MAFAT/crops/train/img/26971.png 0 4 17
/media/gal/USB/MAFAT/crops/train/img/19301.png 0 4 20
/media/gal/USB/MAFAT/crops/train/img/35906.png 1 7 17
/media/gal/USB/MAFAT/crops/train/img/24690.png 1 11 23
/media/gal/USB/MAFAT/crops/train/img/27442.png 0 9 24
/media/gal/USB/MAFAT/crops/train/img/26689.png 1 11 23
/media/gal/USB/MAFAT/crops/train/img/38403.png 0 4 19
/media/gal/USB/MAFAT/crops/train/img/37141.png 0 4 19
/media/gal/USB/MAFAT/crops/train/img/16961.png 0 4 21
/media/gal/USB/MAFAT/crops/train/img/23952.png 1 16 24
This is the solver file:
net: "/home/gal/caffe-1.0/models/mixDCNN/hdf5/AlexNetMulti.prototxt"
test_initialization: false
test_iter: 40
test_interval: 100
base_lr: 0.0001
momentum: 0.9
gamma: 0.1
lr_policy: "step"
stepsize: 1600 # drop the learning rate every .... iterations
max_iter: 4800
display: 100
weight_decay: 0.0005
snapshot: 300000
snapshot_prefix: "/home/gal/caffe-1.0/models/mixDCNN/hdf5/snapshot/GoogleNet_class"
solver_mode: GPU
Why am I keep getting accuracy 0 on "color" class?
You set num_output: 8 for your "fc8_3_color" layer, that is, you are trying to predict "color" with 8 discrete classes 0..7. However, looking at your data the labels for color ranges from 17 to 24 -- this cannot work. the labels MUST be from 0..7 if you are using "SoftmaxWithLoss" and "Accuracy" layers.