Search code examples
python-3.ximage-processingjupyter-notebookchainer

While training epoch details are not updated and does not complete all eochs


I am training blood cell images using chainer. While training the epoch details doesn't get updated and does not run the given set of epochs. I want to understand the cause of this problem..

When the training is interrupted and restarted only a single epoch is updated and displayed..

I am not sure of the reason behind the problem..so I can't point towards a particular section of the code..whether it is the data pre-processing, or data feeding or the classifier/evaluator section.

You can see the whole code here...https://github.com/atom2k17/BloodCell-Chainer/blob/master/WithoutKerasDD-checkpoint.ipynb

After training epoch, main/loss, validation/loss, etc should be populated with values from each epoch..and each epoch should get updated after each epoch is finished.


Solution

  • Can you try modifying

    valid_iter = iterators.SerialIterator(valid, batch_size)

    to

    valid_iter = iterators.SerialIterator(valid, batch_size, repeat=False, shuffle=False)?

    Without repeat=False option, the iterator will not finish so E.Evaluator(valid_iter, model_loss, device=gpu_id) never finish.