pytorch: autograd with tensors generated by arange

Question

I want to compute the gradient of a function in several points. However, If I use tensors generated with torch.arange the gradient is not computed. Instead, using classical tensors it works. Why?

import torch
from torch import tensor

def l(w_1,w_2):
    return w_1*w_2

w_1 = tensor(3., requires_grad=True)
w_2 = tensor(5., requires_grad=True)
l_v = l(w_1, w_2)
l_v.backward()
print(l_v.item(), w_1.grad, w_2.grad) # HERE WORKS OK
#############
for w_1_value  in torch.arange(+2,+4,0.1, requires_grad=True):
    for w_2_value  in torch.arange(-2,+4,0.1, requires_grad=True):
        print(w_1_value, w_2_value)
        l_value = l(w_1_value, w_2_value)
        l_value.backward()
        print(l_value.item(), w_1_value.grad, w_2_value.grad) # HERE I GET NONE ON GRAD VALUES
        

Answer 1

Gradients are computed on leaf tensors, which in this case are the torch.arange tensors themselves. You need to access the grad attribute of those tensors.

When you iterate over the items in the torch.arange tensors, you are getting a view of the tensor, which doesn't have the grad.

This works for what you're trying to do:

w_1_values = torch.arange(+2,+4,0.1, requires_grad=True)
w_2_values = torch.arange(-2,+4,0.1, requires_grad=True)

for w_1_value  in w_1_values:
    for w_2_value  in w_2_values:
        l_value = l(w_1_value, w_2_value)
        l_value.backward()

print(w_1_values.grad)
print(w_2_values.grad)