Search code examples
neural-networkbackpropagation

Multiclass classification and the sigmoid function

Say have a training set Y : 

1,0,1,0
0,1,1,0
0,0,1,1
0,0,1,0

And sigmoid function is defined as :

enter image description here

As the sigmoid function ouputs a value between 0 and 1 does this mean that the training data and value's we are trying to predict should also fall between 0 and 1 ?

Is also correct to use the sigmoid function for making predictions when training set values are not between 0 and 1 ? :

1,4,3,0
2,1,1,0
7,2,6,1
3,0,5,0
Solution 
  • Yes, it is perfectly valid have non binary features.

    The output falls between 0 and 1 because of the nature of the sigmoid function, there is nothing that stops you from having non binary feature set.

    Do the predictions have to be binary?

    Yes, you can have multiclass logistic classification as well.

    The simplest way of doing that is solving a one-vs-all classification problem, wherein you train one binary logistic classifier for each of the labels.

    For example. if your prediction space spans (1, 2, 3, 4), you can have 4 logistic classifiers.

    Given any point in the test set, you can give it the label corresponding to the classifier which is most confident (i.e. has the highest score for that test point).