When I convert a numpy array to tensor and change the numpy in differnet ways some change the tensor and some not. it is sample:
in this code:
array = np.arange(1., 8.)
tensor = torch.from_numpy(array)
array, tensor
the answer is :
(array([1., 2., 3., 4., 5., 6., 7.]),
tensor([1., 2., 3., 4., 5., 6., 7.], dtype=torch.float64))
No when I add 1 to numpy array in this way it change tensor values:
array = np.arange(1., 8.)
tensor = torch.from_numpy(array)
array += 1
array, tensor
The output is:
(array([2., 3., 4., 5., 6., 7., 8.]),
tensor([2., 3., 4., 5., 6., 7., 8.], dtype=torch.float64))
But when I change numpy array in another way it dos not change tensor:
array = np.arange(1., 8.)
tensor = torch.from_numpy(array)
array = array + 1
array, tensor
the output is :
(array([2., 3., 4., 5., 6., 7., 8.]),
tensor([1., 2., 3., 4., 5., 6., 7.], dtype=torch.float64))
again this code dos not change it:
array = np.arange(1., 8.)
tensor = torch.from_numpy(array)
array = array + 1
array += 1
array, tensor
This also happened when changing pytorch's tensor to numpy array.
Why this way work numpy and torch?
+= modifies in place but x = x + 1 creates a copy.
You may want to read about pinned memory. I suspect PyTorch is using it to reduce data transfer overheads for better performance. When you re-assign using array = array + 1 a copy is created (you can try printing the address of the array using id(array) to check if the address is changed after the assignment, meaning a copy is made). When a copy is made, array and tensor no longer point to the same memory address, so modifying array no longer updates the pinned memory.
import numpy as np
arr = np.arange(3)
print((arr, id(arr)))
arr+=1
print((arr, id(arr)))
arr = arr + 1
print((arr, id(arr)))
which (for me) results in:
(array([0, 1, 2]), 1951948494416)
(array([1, 2, 3]), 1951948494416)
(array([2, 3, 4]), 1952052860880)
When you create a copy, the old array is still being used by the GPU so it remains there (memory isn't freed) so you can still access its old content (check pass by reference vs pass by value).