Search code examples
pythonmathsympyderivativesigmoid

The right way to calculate the derivative of sigmoid function in Python

I was walking through this article. In the code, the author mentions that the following function finds the derivative:

# convert output of sigmoid function to its derivative
def sigmoid_output_to_derivative(output):
    return output*(1-output)

I couldn't really understand how the derivative was found here. Would using SymPy to find the derivative be a better option? How can I do it in the example shown in the article, especially that output will be a list similar to the following:

[[ 0.44856632  0.51939863  0.45968497  0.59156505]
 [ 0.28639589  0.32350963  0.31236398  0.51538526]
 [ 0.40795614  0.62674606  0.23841622  0.49377636]
 [ 0.25371248  0.42628115  0.14321233  0.41732254]]

So, the bottom line is:

  • How was the derivative calculated in the original article? I understand the formula in the code, but, is there some theoretical basis?
  • Can we use another way to find the derivative? A more clear way maybe?

Thanks.

Solution

  • The sigmoid function is useful mainly because its derivative is easily computable in terms of its output; the derivative is f(x)*(1-f(x)).

    Therefore, finding the derivative using a library based on the sigmoid function is not necessary as the mathematical derivative (above) is already known. For the derivation, see this.