Encog - EarlyStoppingStrategy with validation set

Question

I would like to stop training a network once I see the error calculated from the validation set starts to increase. I'm using a BasicNetwork with RPROP as the training algorithm, and I have the following training iteration:

double validationError = 999.999;


    while(!stop){

        train.iteration(); //weights are updated here

        System.out.println("Epoch #" + epoch + " Error : " + train.getError()) ;

        //I'm just comparing to see if the error on the validation set increased or not
        if (network.calculateError(validationSet) < validationError)
            validationError = network.calculateError(validationSet); 

        else
            //once the error increases I stop the training.
            stop = true ;

        System.out.println("Epoch #" + epoch + "Validation Error" + network.calculateError(validationSet));

        epoch++;

    }
        train.finishTraining();

Obviously this isn't working because the weights have already been changed before figuring out if I need to stop training or not. Is there anyway I can take a step back and use the old weights?

I also see the EarlyStoppingStrategy class which is probably what I need to use by using the addStrategy() method. However, I really don't understand why the EarlyStoppingStrategy constructor takes both the validation set and the test set. I thought it would only need the validation set and the test set shouldn't be used at all until I test the output of the network.

Answer 1

Encog's EarlyStoppingStrategy class implements an early stopping strategy according to this paper:

Proben1 | A Set of Neural Network Benchmark Problems and Benchmarking Rules

(a full cite is included in the Javadoc)

If you just want to stop as soon as a validation set's error no longer improves you may just want to use the Encog SimpleEarlyStoppingStrategy, found here:

org.encog.ml.train.strategy.end.SimpleEarlyStoppingStrategy

Note, that SimpleEarlyStoppingStrategy requires Encog 3.3.