How can I describe a network for function approximation with CNTK BrainsScript?

Question

I want a network that approximates arbitrary functions. Here, I assume that this network has one input, (n) hidden layers, each layer have (n) nodes, and one output.

I use CNTK config file mainly. How can I describe such a network in CNTK config file?

I've been trying it out with SimpleNetworkBuilder like below and using sine wave training data.

Here is the config file and training data. Please refer it.

https://drive.google.com/open?id=0B83LnG3hRTNGUGxvTHJfYmZuMEE

To evaluating this network, I slightly modified an example project in CNTK package, CNTKLibraryCPPEvalGPUExamples. I've only put 'EvalMultithreads.cpp' in above zip file.

This network begins learning iteration but it outputs wrong values on my evaluation process(I've been using C++ Eval Library).

I gave a few data to this evaluation program and it outputs '1' on all outputs. This network obviously learnt wrong.

command = trainNetwork:testNetwork
precision = "float"; 
traceLevel = 1; 
deviceId = 0;

rootDir = "."; 
dataDir = ".";
outputDir = "./Output";

modelPath = "$outputDir$/Models/mynn2"

dimension = 1
labelDimension = 1

# TRAINING CONFIG
trainNetwork = {
    action = "train"

    SimpleNetworkBuilder = [
        layerSizes        = 1:50*1:1
        trainingCriterion = "CrossEntropyWithSoftmax"
        evalCriterion     = "ErrorPrediction"
        layerTypes        = "Sigmoid"
        applyMeanVarNorm  = true
    ]

    SGD = [
        epochSize = 10000
        minibatchSize = 1
        learningRatesPerSample = 0.0001
        momentumAsTimeConstant = 0.0
        maxEpochs = 1
    ]

    reader = {
        readerType = "CNTKTextFormatReader"
        file = "train_sine.txt"
        input = {
            features = { dim = $dimension$; format = "dense" }
            labels =   { dim = $labelDimension$; format = "dense" }
        }
    }
}

# TEST CONFIG
testNetwork = {
    action = "test"
    minibatchSize = 1    # reduce this if you run out of memory

    reader = {
        readerType = "CNTKTextFormatReader"
        file = "test.txt"
        input = {
            features = { dim = $dimension$; format = "dense" }
            labels =   { dim = $labelDimension$; format = "dense" }
        }
    }
}

Answer 1

When doing regression you should use SquaredError not CrossEntropyWithSoftmax.