Perceptron Learning Algorithm diverging

Question

This is my first time writing a perceptron learning algorithm from scratch. I've used out of the box ML solutions before, but wanted to really understand it and write it myself. For some reason my error rate keeps increasing rather than decreasing. So it appears that my algorithm is diverging instead of converging. I wrote in a tolerance range because it would get close sometimes but never quite hit the mark.

I have three weights; 1 for bias and 2 for X and Y, respectively.

I find my discriminant with: D = (weight0 + weight1 * Xi) + (weight2 * Yi)

If the discriminant doesn't match the expected output, then I update the weights with: Note: Assume c and k are pre-defined constants and d = expected output w0 = w0 + cdk, w1 = w1 + cdXi, w2 = w2 + cdYi

Below is my implementation in Python:

def weightsUpdate(weights, constantC, constantK, classificationd, x, y):
     weights[0] = weights[0] + constantC * classificationd * constantK # w0 = w0 + cdk
     weights[1] = weights[1] + constantC * classificationd * x #w1 = w1 + cdx
     weights[2] = weights[2] + constantC * classificationd * y #w2 = w2 + cdy

     return weights

def trainModel(df, weights, constantC, constantK, maxIter, threshHold):
     #grab the values in a list
     x = df['X'].values
     y = df['Y'].values
     d = df['Class'].values

     #define some variables to keep track
     numTurns = 0

     while numTurns < maxIter:
          errorRate = 0
          falsePosNeg = 0
          truePosNeg = 0

          '''assign som threshhold values. must accomodate for slight variance.'''
          posThreshHoldCeiling = 1 + threshHold
          posThreshHoldFloor = 1 - threshHold
          negThreshHoldFloor = -1 - threshHold
          negThreshHoldCeiling = -1 + threshHold

          for i in range(len(x)):
              ''' calculate the discriminant D = w0 + w1*xi + w2*yi '''
              discriminant = weights[0] + (weights[1] * x[i]) + (weights[2] * y[i])

              '''if the discriminant is not correct when compared to the correct output'''
              if ((discriminant >= posThreshHoldFloor and discriminant <= posThreshHoldCeiling) or
                (discriminant >= negThreshHoldFloor and discriminant <= negThreshHoldCeiling)):
                  truePosNeg += 1
                 #weights = weightsUpdate(weights, constantC, constantK, d[i], x[i], y[i])
        else:
                 '''update the weights'''
                 weights = weightsUpdate(weights, constantC, constantK, d[i], x[i], y[i])
                 falsePosNeg += 1


          numTurns += 1 #increase number of turns by 1 iteration
          print("Number of False Positive/Negative: " + str(falsePosNeg))
          print("Number of True Positive/Negative: " + str(truePosNeg))
          errorRate = falsePosNeg / len(x) * 100
          print("Error rate: " + str(errorRate) + "%")


         '''add stop conditions'''
         if (errorRate < 25):
             break
         else:
             continue

Thank you for any and all help.

Answer 1

The Perceptron Learning Algorithm (PLA) should not need a threshold. Also PLA requires that only the signs of the discriminant and expected output should match, i.e., sign(Discriminant) == d, so there is no need to converge on the actual output. Below is a modified version of trainModel().

def trainModel(df, weights, constantC, constantK, maxIter):
#grab the values in a list
x = df['X'].values
y = df['Y'].values
d = df['Class'].values
globalErrorRate = 0

#define some variables to keep track
numTurns = 0

while numTurns < maxIter:
    localErrorRate = 0
    successRate = 0
    falsePosNeg = 0
    truePosNeg = 0

    for i in range(len(x)):
        #calculate the discriminant
        discriminant = weights[0] + (weights[1] * x[i]) + (weights[2] * y[i])

        if(isPos(discriminant) and d[i] == 1):
            truePosNeg += 1

        elif(isPos(discriminant) == False and d[i] == -1):
            truePosNeg += 1

        else:
            falsePosNeg += 1
            weights = weightsUpdate(weights, constantC, constantK, d[i], x[i], y[i])

    numTurns += 1 #increase number of turns by 1 iteration
    print("Number of False Positive/Negative: " + str(falsePosNeg))
    print("Number of True Positive/Negative: " + str(truePosNeg))
    localErrorRate = falsePosNeg / len(x) * 100
    successRate = truePosNeg / len(x) * 100
    print("Error rate: " + str(localErrorRate) + "%")
    print("Success rate: " + str(successRate) + "%")

    if successRate == 100:
        print("Trained Weight Values: " + str(weights))
        break
    else:
        continue