Why is this AND gate neural network not moving towards optimal weights?

Question

I have simple neural network of two inputs and one output with no hidden layers. i.e,

[input1][weight1 weight2] = z [input2]
output = sigmoid(z)

The weights don't seem move to an optimal value. I've checked the gradient to the best of my knowledge and I can see the weights go up or down depending on the derivative of cost function, but network doesn't move towards optimal values.

Here's the code:

import numpy as np
import random as r
import sys

def sigmoid(ip, derivate=False):
    if derivate:
        return ip*(1-ip)
    return 1.0/(1+np.exp(-1*ip))

class NeuralNet:
    global sigmoid 

    def __init__(self):
        self.inputLayers = 2
        self.outputLayer = 1

    def setup(self):
        self.i = np.array([r.random(), r.random()], dtype=float).reshape(2,)
        self.w = np.array([r.random(), r.random()], dtype=float).reshape(2,)

    def forward_propogate(self):
        self.z = self.w*self.i
        self.o = sigmoid(sum(self.z))

    def optimize_cost(self, desired):
        i=0
        current_cost = pow(desired - self.o, 2)
        for weight in self.w:
            dpdw = -1 * (desired-self.o) * (sigmoid(self.o, derivate=True)) * self.i[i]
            print(dpdw)
            self.w[i] = self.w[i] + 500*dpdw
            i+=1
        self.forward_propogate()

    def train(self, ip, op):
        self.i = np.array(ip).reshape(2,)
        self.forward_propogate()
        print("before:{}".format(self.o))
        self.optimize_cost(op[0])
        # print(self.i,self.w)
n = NeuralNet()
n.setup()
# while sys.stdin.read(1):
while True:
    a = r.random()
    b = r.random()
    if a>0.5 and b>0.5:
        c = 0.9
    else:
        c = 0.1
    print(c)
    n.train([a,b],[c])
    print(n.i, n.w)
    print("after: {}".format(n.o))

Answer 1

Answering my own question. All I needed was a BIAS. Without a BIAS, the sigmoid cannot deviate from 0.

Here's a sigmoid with bias of 2. Now sigmoid(0) = closer to 0.1

After including a BIAS node in the network, I was able to get the results.

Success rate: 99.00000042272556% Network trained, took: 2365601 trials
Network weights:[14.0435016 14.04351048]
Bias: 21.861074330808844

---

Enter INPUT:
0
1
Network output:0.00040243926180320134

Enter INPUT:
1
1
Network output:0.9980264340845117