Perceptron - AND operator

Question

I used this code to implement AND operator, but line that separates regions is not properly calculated. It is going through points [1, 0] and [0, 1]. What to do in order to correctly separate regions?

from sklearn.linear_model import Perceptron
import matplotlib.pyplot as plt
import numpy as np
from itertools import product

data = [[0,0],[1,0],[0,1],[1,1]]
labels_and = [0, 0, 0, 1]

x = [points[0] for points in data]
y = [points[1] for points in data]

plt.scatter(x, y, c = labels_and)
plt.show()

classifier = Perceptron()
classifier.fit(data, labels_and)

print(classifier.score(data, labels_and))

result = classifier.decision_function([[0, 0], [1, 1], [0.5, 0.5]])
print(result)

x_values = y_values = np.linspace(0, 1, 100)
point_grid = list(product(x_values, y_values))
distances = classifier.decision_function(point_grid)
abs_distance = [abs(x) for x in distances]

distance_matrix = np.reshape(abs_distance, (100, 100))

heatmap = plt.pcolormesh(x_values, y_values, distance_matrix)
plt.colorbar(heatmap)
plt.show()

Screenshot here

Answer 1

The decision boundary is correct because everything > is classified as class 1, and everything <= as class 0. However your way of visualizing the absolute value of the out come of the classifier prediction is maybe misleading?

---

The sklearn docs refer to the English wiki article which uses the following definition for a perceptron:

Since the only class 1 label is the point (1,1), the upper right hand corner we can choose i.e. w = [2,2] and b=-2 to correctly classify (1,1) as 1 and (0,0), (0,1), (1,0) as 0.

The points (1,0) and (0,1) will lie on the decision boundary since 2*0+2*1-2 =0. However we also could have chosen b=-3 and our classification problem would still be solved correctly. The difference is that the points (1,0) and (0,1) no longer lie directly at the boundary since 2*0+2*1-3 <0.

Lets see what parameters our trained perceptron learned! To access them you can see from the sklearn docs

w = classifier.coef_[0]  
b = classifier.intercept_   

If you print them you probably obtain w=[2,2] and b=-2. To derive the line representing the according decision boundary you can consider the edge cases where w0*x + w1*y + b == 0 and solve for x. You will obtain:

So plotting this will lead to something like this:

m = -weight[0]/weight[1]   
n = -b/weight[1]    
plt.plot(x_values, m*x_values+n)   

which is the dark blue valley you are seeing in your picture. Also our points (1,0) and (0,1) lie on the boundary. You can play around with b=-3 where you obtain a line with a higher y-intercept which you are probably expecting?