Search code examples
mathmachine-learningsymbolic-mathlogistic-regression

How to represent hypothesis function in logistic regression cost function

Below is logistic regression cost function with features(x) , training examples(y)

enter image description here

How should the hypotheses function (circled red) be represented ? : enter image description here

I'm attempting to implement this function but unsure what value (or function) the hypothesis should take ?

Solution

  • The activation function in logistic regression is the sigmoid function (https://en.wikipedia.org/wiki/Sigmoid_function), defined as

    enter image description here

    which is also probability of y taking on a 1 value for a given x and parameter theta's to be determined (sigmoid is always between 0 and 1)

    The cost function you mentioned comes from maximum likelihood estimation (https://en.wikipedia.org/wiki/Maximum_likelihood) of training (X, y) pairs. The log-likelihood of any (X, y) pair is exactly

    enter image description here

    The final loss function is precisely the sum of all log-likelihood for all (X, y) training pairs.

    Thus, the ''hypothesis" you are talking about is simply sigmoid, 1/(1+exp(-theta * x)) (actually I am not familiar with the term hypothesis used in this context, but the expression resembles any standard expression involving sigmoid and MLE)