How to monitor error on a validation set in Chainer framework?

Question

I am kind of new to Chainer and have written a code which trains a simple feed forward neural network. I have a validation set and a train set and want to test on the validation set on each like 500 iterations and if the results are better I want to save my network weights. Can anyone tell me how can I do that?

Here is my code:

optimizer = optimizers.Adam()
optimizer.setup(model)

updater = training.StandardUpdater(train_iter, optimizer, device=0)
trainer = training.Trainer(updater, (10000, 'epoch'), out='result')

trainer.extend(extensions.Evaluator(validation_iter, model, device=0))
trainer.extend(extensions.LogReport())
trainer.extend(extensions.PrintReport(['epoch', 'main/loss',  'validation/main/loss', 'elapsed_time']))
trainer.run()

Answer 1

1. Error on validation set

It is reported by Evaluator, and printed by PrintReport. Thus it should be shown with your code above. And to control the frequency of execution of these extentions, you can specify trigger keyword argument in trainer.extend function. For example, below code specifies printing each 500 iteration.

trainer.extend(extensions.PrintReport(['epoch', 'main/loss', 'validation/main/loss', 'elapsed_time']), trigger=(500, 'iteration'))

You can also specify trigger to Evaluator.

2. Save network weights

You can use snapshot_object extension.

http://docs.chainer.org/en/stable/reference/generated/chainer.training.extensions.snapshot_object.html

It will be invoked every epoch as default.

If you want to invoke it when the loss improves, I think you can set trigger using MinValueTrigger.

http://docs.chainer.org/en/stable/reference/generated/chainer.training.triggers.MinValueTrigger.html