I have been tinkering with standard Multi-Layer Perceptrons and the Backpropagation algorithm in Encog for two weeks now, both via workbench and via Java code. My next job will require inserting noise in input patterns, like in this paper: http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=6033567 (PCA and Gaussian noise in MLP neural network training improve generalization in problems with small and unbalanced data sets)
Basically, I need to (it is a binary classification problem): 1 - Transform the input patterns using Principal Component Analysis (PCA) 2 - Use Backpropagation to train an MLP, with a trick: Insert a different white noise in each training pattern on each epoch.
What is the more straightfoward way to do this noise injection using the Java version of Encog? Does any of the available training algorithms involve artificial noise injection?
PS.: The full algorithm for the paper I cited is
1. Apply PCA to decorrelate the variables
2. Initialize the system architecture
3. Set k, max number of epochs and min error
4. Begin training - While epoch counter
a. Randomly draw an input pattern (vector x) without replacement for presentation
b. Inject noise into input pattern
1. For every variable from the input pattern
a. Draw g from a Gaussian distribution. g ~ N(0,1)
b. Calculate n = k * g
c. Add ninto input pattern x
c. Present the input pattern
d. Adjust the system parameters
e. If training stopping criterion has been reached then
1. Stop training
f. Otherwise
1. Increment epoch counter
2. Go to 4.a
I think the best way to do this is going to be to create a class that implements the MLDataSet interface. You would then provide a regular BasicMLDataSet (or other data set) to your new version of MLDataSet. For the size() method you would return the number of training patterns that you want trained per iteration. Then for each call to the to your new MLDataSet to return a MLDataPair you randomly select a pair from the provided data set, you would then clone this element, and add the noise as described, and return it.
Does that sounds like it would accomplish what the paper is describing? If you end up implementing this and would like to contribute it to the Encog project, that would be great. I might attempt it myself as well, if you do not.