So I'm using Jeff Heaton's Neural Network library.
When trying to solve the Iris plant classification problem I have an issue with data normalization.
I am able to Normalize a CSV file using the following method:
public void NormalizeFile(FileInfo SourceDataFile, FileInfo NormalizedDataFile, FileInfo NormalizationConfigFile)
{
var wizard = new AnalystWizard(_analyst);
wizard.Wizard(SourceDataFile, _useHeaders, AnalystFileFormat.DecpntComma);
var norm = new AnalystNormalizeCSV();
norm.Analyze(SourceDataFile, _useHeaders, CSVFormat.English, _analyst);
norm.ProduceOutputHeaders = _useHeaders;
norm.Normalize(NormalizedDataFile);
// save normalization configuration, which can be used later to denormalize to get the raw output.
_analyst.Save(NormalizationConfigFile);
}
So far so good... The program works with a high degree of accuracy.
The problem occurs when I want to enter the values into my console application.
I have some input data
Each of these values has a different high/low I would like to normalize these values so that I can feed them into my network without writing a CSV file to disk.
I later realised that what I really needed as an analyser that would all me to automatically normalise a mixture of qualitative (nominal) and quantitive data (just like the CSV implementation).
The problem was that the existing code was tightly coupled to CSV files. To combat this I wrote my own encog extension method library.
it can be found here: https://github.com/KiransHub/encog-dotnet-core
Here's an example of it in action:
public void NormalizeDataExample()
{
List<LoadedMarketData> AppleMarketData = GetMarketData("AAPL");
List<LoadedMarketData> MicrosoftMarketData = GetMarketData("MSFT");
List<LoadedMarketData> YahootMarketData = GetMarketData("YHOO");
List<LoadedMarketData> MarketData = new List<LoadedMarketData>();
MarketData.AddRange(AppleMarketData);
MarketData.AddRange(MicrosoftMarketData);
MarketData.AddRange(YahootMarketData);
DataSet dataSet = new DataSet().Convert(MarketData, "Market DataSet");
var analyst = new EncogAnalyst();
var wizard = new AnalystWizard(analyst);
wizard.Wizard(dataSet);
var normalizer = new AnalystNormalizeDataSet(analyst);
var normalizedData = normalizer.Normalize(dataSet);
}