FANN doesn't train properly

Question

I am trying to approximate the square function with FANN. The code follows:

#include "../FANN-2.2.0-Source/src/include/doublefann.h"
#include "../FANN-2.2.0-Source/src/include/fann_cpp.h"
#include <cstdlib>
#include <iostream>

using namespace std;
using namespace FANN;

//Remember: fann_type is double!
int main(int argc, char** argv) {
    //create a test network: [1,2,1] MLP
    neural_net * net = new neural_net;
    const unsigned int layers[3] = {1,3,1};
    net->create_standard_array(3,layers);

    //net->create_standard(num_layers, num_input, num_hidden, num_output);

    net->set_learning_rate(0.7f);

    net->set_activation_steepness_hidden(0.7);
    net->set_activation_steepness_output(0.7);

    net->set_activation_function_hidden(SIGMOID_SYMMETRIC_STEPWISE);
    net->set_activation_function_output(SIGMOID_SYMMETRIC_STEPWISE);
    net->set_training_algorithm(TRAIN_QUICKPROP);

    //cout<<net->get_train_error_function()
    //exit(0);
    //test the number 2
    fann_type * testinput = new fann_type;
    *testinput = 2;
    fann_type * testoutput = new fann_type;
    *testoutput = *(net->run(testinput));
    double outputasdouble = (double) *testoutput;
    cout<<"Test output: "<<outputasdouble<<endl;

    //make a training set of x->x^2
    training_data * squaredata = new training_data;
    squaredata->read_train_from_file("trainingdata.txt");

    net->train_on_data(*squaredata,1000,100,0.001);

    cout<<endl<<"Easy!";
    return 0;
}

trainingdata.txt is this:

10 1 1
1 1
2 4
3 9
4 16
5 25
6 36
7 49
8 64
9 81
10 100

I feel that I have done everything right with the API. However, when I run it, I get huge error that never seems to reduce with training.

Test output: -0.0311087
Max epochs     1000. Desired error: 0.0010000000.
Epochs            1. Current error: 633.9928588867. Bit fail 10.
Epochs          100. Current error: 614.3250122070. Bit fail 9.
Epochs          200. Current error: 614.3250122070. Bit fail 9.
Epochs          300. Current error: 614.3250122070. Bit fail 9.
Epochs          400. Current error: 614.3250122070. Bit fail 9.
Epochs          500. Current error: 614.3250122070. Bit fail 9.
Epochs          600. Current error: 614.3250122070. Bit fail 9.
Epochs          700. Current error: 614.3250122070. Bit fail 9.
Epochs          800. Current error: 614.3250122070. Bit fail 9.
Epochs          900. Current error: 614.3250122070. Bit fail 9.
Epochs         1000. Current error: 614.3250122070. Bit fail 9.

Easy!

What have I done wrong?

Answer 1

If you use sigmoid function for your output layer, the output will provide the range of (0,1).

You may have two choice, (1) divide all your output by a constant, say, 1e4. When a test data comes, you also divide it by 1e4. The problem is that you may not predict the square number larger than 100 (100^2=1e4);(2) make both hidden and output layer to the linear, and the network will automatically learn the weights to give whatever output values you have.