Neural Networks Using Python and NumPy

Question

I am newbie to NN and I am trying to implement NN with Python/Numpy from the code I found at: "Create a Simple Neural Network in Python from Scratch" enter link description here

My input array is:

array([[5.71, 5.77, 5.94],
   [5.77, 5.94, 5.51],
   [5.94, 5.51, 5.88],
   [5.51, 5.88, 5.73]])

Output array is:

array([[5.51],
   [5.88],
   [5.73],
   [6.41]])

after running the code, I see following results which are not correct:

synaptic_weights after training
[[1.90625275]
[2.54867698]
[1.07698312]]
outputs after training
[[1.]
[1.]
[1.]
[1.]]

Here is the core of the code:

for iteration in range(1000):
    input_layer = tr_input
    outputs = sigmoid(np.dot(input_layer, synapic_weights))

    error = tr_output - outputs

    adjustmnets = error * sigmoid_derivative(outputs)

    synapic_weights +=np.dot(input_layer.T, adjustmnets )

print('synaptic_weights after training')  
print(synapic_weights)

print('outputs after training')  
print(outputs)

What should I change in this code so it works for my data? Or shall I take different method? Any help is highly appreciated.

Answer 1

That's because you are using a wrong activation function (i.e. sigmoid). The main reason why we use sigmoid function is because it exists between (0 to 1). Therefore, it is especially used for models where we have to predict the probability as an output.Since probability of anything exists only between the range of 0 and 1, sigmoid is the right choice.

If you want to train a model to predict the values in your array, you should use a regression model. Otherwise, you can convert your output into labels (for example are 5.x to 0 and 6.x to 1) and retrain your model.