Search code examples
machine-learningtheanologistic-regression

Logistic function Addition or subtraction

I'm currently studying Machine Learning but I don't have a statistics background. Everywhere I've seen the logistic function, it has always been:

wx + b

but this example in Theano documentation used:

wx - b

Please which one is it? I'm new to this and I don't want to get confused.

Solution

  • The example on your linked page is not using wx - b. Here is the formula I assume you are referencing:

    p_1 = 1 / (1 + T.exp(-T.dot(x, w) - b))

    You can break this up into the sigmoid argument and the sigmoid function:

    arg = T.dot(x, w) + b         # sigmoid argument
p_1 = 1 / (1 + T.exp(-arg))   # sigmoid function

    So there are two issues. The first is that you didn't factor the sign of the b variable properly (the formula is using wx + b). Second is that the formula you quoted isn't actually the sigmoid function; rather, it is the argument (a linear weighted sum of input variables) that is passed to the sigmoid function.