Difference between v.assign(v + 1) and v = v + 1 in Tensorflow

Question

The following Tensorflow code works fine and v1 becomes [1., 1., 1.]

v1 = tf.get_variable('v1', shape=[3], initializer=tf.zeros_initializer)
v1 = v1 + 1 

with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    print (v1.eval())

The following code segment also gives us the exactly the same result as the above. v1 becomes [1., 1., 1.] if we run sess.run(inc_v1).

v1 = tf.get_variable('v1', shape=[3], initializer=tf.zeros_initializer)
inc_v1 = v1.assign(v1 + 1)


with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    sess.run(inc_v1)
    print (v1.eval())

However, the following code causes an error.

v1 = tf.get_variable('v1', shape=[3], initializer=tf.zeros_initializer)
v1 = v1 + 1 
inc_v1 = v1.assign(v1 + 1)


with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    sess.run(inc_v1)
    print (v1.eval())

The error is as follows:

AttributeError: 'Tensor' object has no attribute 'assign'

Could you please tell me why it causes an error?

Answer 1

Tensors and Variables are different objects in TensorFlow

import tensorflow as tf


def inspect(t):
    print('\n %s\n-------' % t.name)
    print(type(t))
    print(t.op.outputs)
    print('has assign method' if 'assign' in dir(t) else 'has no assign method')


v1 = tf.get_variable('v1', shape=[3], initializer=tf.zeros_initializer)
inspect(v1)
v2 = v1 + 1
inspect(v2)

gives

 v1:0
-------
<class 'tensorflow.python.ops.variables.Variable'>
[<tf.Tensor 'v1:0' shape=(3,) dtype=float32_ref>]
has assign method

 add:0
-------
<class 'tensorflow.python.framework.ops.Tensor'>
[<tf.Tensor 'add:0' shape=(3,) dtype=float32>]
has no assign method

Hence, v1:0 is really the variable itself and v1 has the method assign. This makes sense because it is just a reference to a float value. On the other hand, v2 = v1 + 1 results in the output of the add operation. So v2 is not a variable anymore and you cannot assign a new value to v2. Which operand of add do you expect to be updated in this case? Whenever you use v1 think of the read_value() operation from v1 being used:

v1 = tf.get_variable('v1', shape=[3], initializer=tf.zeros_initializer)
inspect(v1)
w = v1.read_value()
inspect(w)
v2 = v1.read_value() + 1
inspect(v2)

gives

 v1:0
-------
<class 'tensorflow.python.ops.variables.Variable'>
[<tf.Tensor 'v1:0' shape=(3,) dtype=float32_ref>]
has assign method

 read:0
-------
<class 'tensorflow.python.framework.ops.Tensor'>
[<tf.Tensor 'read:0' shape=(3,) dtype=float32>]
has no assign method

 add:0
-------
<class 'tensorflow.python.framework.ops.Tensor'>
[<tf.Tensor 'add:0' shape=(3,) dtype=float32>]
has no assign method