I have a question about automatic differentiation and especially in Pytorch since i am using this library.I have seen for instance automatic differentiation give the partial derivatives of an expression with respect to a variable.
However,as far as what i have seen ,the result is always given at a specific point,meaning it is a tensor with numerical values.My question is the following: let's say we define a function of two variables: f(x,y)= x² + y² .
Is Pytorch able to return a function which corresponds to the partial derivative of f with respect to x or y? That is to return the following definitions:
> def partial_f_x:
return 2*x
def partial_f_y:
return 2*y
Because even though the function f here is rather simple,it would be interesting if Pytorch could give us a formula(depending on the different variables) of the derivatives,instead of giving a numerical value at a given point,because in that case we do not know the expression of the derivatives.
So if I summarize: is Pytorch able to return formulas for derivatives of complicated functions? or does it just return a tensor with numerical values for the derivative at a given point?
Thank you so much!
That is not how pytorch is doing to get the derivative. Most (probably all) of the computational packages are using approximation methods to get the derivative value rather than deriving the derivative function, so they don't care what is the derivative in mathematical terms.
If you are looking for something like that, you can try to use the sympy library for symbolic mathematics. Here's an example:
import sympy as sym
x = sym.Symbol('x')
y = sym.Symbol('y')
sym.diff(x**2 + y**2, x, 1)
# => 2*x
sym.diff(x**2 + y**2, y, 1)
# => 2*y
Then to evaluate, you can simply substitute in the values you want to use for the variable(s):
dfdx.subs(y,1)
# => 2