Being new to deep learning world and after reading a lot of theories, I was trying to understand how does the neural net learn practically, so I provided it with simple dataset where input columns are [[x,x+y]...] and output column [[x/(x+y)]...] and tried it using tflearn but even after countless tries(2 days) the network is not able to minimize loss. Even after it minimizes(after adding tanh layer) the network predictions are way off. Can someone help me with this? Below is the code.
neural_net = tflearn.input_data(shape=[None, 2])
neural_net = tflearn.fully_connected(neural_net, 1,
activation='linear', bias=True)
neural_net = tflearn.regression(neural_net, optimizer='sgd',
learning_rate=0.001, loss='mean_square')
# train
model = tflearn.DNN(neural_net,tensorboard_verbose=3)
model.fit(OR, Y_truth, n_epoch=1000,
snapshot_epoch=False,validation_set=0.1,batch_size=10)
print model.get_weights(neural_net.W)
# prediction
print model.predict([[2,2]])
The output prediction is 0.13946463!!
Your network won't be able to learn that function.
The network has a single neuron. That's equivalent to ax+by+c where you try to learn a,b,c.
You need more non-linearity in your model. You need to add more layers and neurons with non-linear activation for the model to be able to learn your desired function.