Unexpected value of numerical gradient

Question

I want to make a function that - returns a vector of the gradient of a function at a specific point

what I tried -

import numpy as np


def function_2(x):
    return x[0]**2 + x[1]**2

def numerical_gradient(f, x):
    h = 1e-4
    grad = np.zeros_like(x)
    for idx in range(x.size):
        tmp_val = x[idx]
        # f(x + h) 
        x[idx] = tmp_val + h
        fxh1 = f(x)

        # f(x - h) 
        x[idx] = tmp_val - h
        fxh2 = f(x)
    
        grad[idx] = (fxh1 - fxh2) / (2 * h)
        #  x[idx] original value
        x[idx] = tmp_val
    return grad

grd1 = numerical_gradient(function_2, np.array([3,4]))
print(grd1)

and it showed [25000 35000] Why it is not [6.000xx 7.999x] as expected??

Answer 1

You need to ensure your numpy arrays are float, not int. One easy way is to define np.array([3.,4.]) instead of np.array([3,4]).

Explanation:

With np.array([3,4]) your array will have dtype=int. When you then do x[idx] = tmp_val - h you effectively try to push e.g. 2.999 into an int array, you can't do that and numpy will automatically truncate it to 2.