Search code examples
matlabneural-networkperceptron

Understanding Perceptron training algorithm

The following text is from Hal Daumé III's "A Course in Machine Learning" online text book (Page-41).

enter image description here

I understand that, D = size of the input vector.

(1) What kind of Perceptron algorithm is this? Binary/Multi-class? Online/Offline?

(2) What is y here? Bias/weight/Sample/class_label?

(3) What is the rationale of testing ya<=0 for updating weights?

EDIT.

y is class_label.

Solution

  • Answers to your questions:

    1 - This is a binary perceptron algorithm, working on an offline batch.

    2 - as you wrote - Y is the labels vector. each label can be either be 1 or -1.

    3 - The rational of testing if y*a<=0 is to check if the perceptron classified a certain sample correctly. If not - the weights of the perceptron are modified.

    A bit more about the 3rd question

    The idea behind the perceptron algorithm is as follows:

    a. we iterate over the samples MaxIter times.

    b. The perceptron classifies each sample by multiplying it with the weights vector W and adding a bias b. The result is assigned into the variable a.

    c. The prediction for each sample can be either 1 or -1. It is calculated by sign(a). At this stage we check the correctness of the classification.

    if y*a>0 that means that y=sign(a). In other words the predicted classification is correct, and we move on to the next sample.

    If however y*a<=0, that means that the perceptron failed to predict the correct label. In this case, the algorithm changes the perceptron's weights in a way that they'll be more compatible to the sample which we failed to classify.