Search code examples
matlabneural-networkoctavegradientgradient-descent

Gradient function on a matrix in Octave/MatLab


I'm trying to implement the gradient descent algorithm in Octave/Matlab. I'm at the point where I have this 201x201 matrix called errors, which I would assume corresponds to a 2 input variables function f(x, y). The matrix gives a nice gradient image when displayed with imagesc, but I am confused as to when I calculate [dx, dy] = gradient(errors). I obtain both dx and dy to be 2 dimensional matrices (201x201) instead of simple vectors. I would assume that, since we calculate the partial derivative in regards to x (resp. y), y (resp. x) so it would disappear from the result of the operation. I'm pretty sure I'm missing something, although I feel like I have a good enough understanding of how the gradient of a function works. Thank you in advance for you answer.

enter image description here


Solution

  • The gradient exists at a point. Your gradient expression is evaluating the (numerical) gradient at all 201x201 points.

    So for example, the gradient of errors at the point (3,4) is the vector [dx(3,4), dy(3,4)].

    This example might help: https://www.mathworks.com/help/matlab/ref/gradient.html#bvhqkfr Notice how the information returned by gradient is enough to plot the whole vector field of gradients.