Pytorch shows the 'NoneType' object has no attribute 'zero_' backward error

Question

Hi i started learning pytorch. When i try to reset weight and bias grad i got 27 b.grad.zero_()

AttributeError: 'NoneType' object has no attribute 'zero_' error.

input = torch.tensor([73,67,43,91,88,64,87,134,58,102,43,37,69,96,70],dtype=torch.float32)
inputnew = input.reshape(5,3)
target = torch.tensor([56,70,81,101,119,133,22,37,103,119])
targetnew = target.reshape(5,2)

w = torch.randn(2,3,requires_grad = True)
b = torch.randn(2,requires_grad = True)

def model(x):
       return x @ w.t()+ b
islem= model(inputnew)
def fix(real,pred):
    diff = real-pred
    return torch.sum((diff*diff)/diff.numel())
loss = fix(targetnew,islem)

for i in range(100):
    islem = model(inputnew)
    loss = fix(targetnew,islem)
    loss.backward()
    with torch.no_grad():
            w=w- w.grad*1e-5 
            b=b-b.grad*1e-5
            w.grad.zero_()
            b.grad.zero_()

How can i solve? Thanks

Answer 1

w = w - w.grad * 1e-5 creates a new tensor and assigns it to w. This breaks the link, resulting in w.grad being set to None.

To fix it you should instead update the tensor in place using -= operator as w -= w.grad * 1e-5:

    with torch.no_grad():
        w -= w.grad * 1e-5
        b -= b.grad * 1e-5
        w.grad.zero_()
        b.grad.zero_()