Search code examples
pythontensorflowgraphvariable-assignmentassignment-operator

Tensorflow: what is the difference between tf.identity and '=' operator

I'm confused about '=' operator, and tf.identity(), I thought '=' is to just make a reference of the tensor, and identity is to make a copy, e.g., with

ref = x
ref = ref*0
sess.run(x)

I will get x all been set to 0 element-wise, and with 

copy = tf.identity(x)
copy = ref*0
sess.run(x)

x would not be changed, since identity make copy, not a reference, but with experiment, '=' also make a copy and x is not set to 0, so what's the difference?

Solution

  • The difference is only in tensorlfow graph layout. tf.identity creates a new op in the graph that mimics its argument, while pure assignment adds a new python variable that points to the same op.

    In both cases (the pair ref and x or the pair copy and x), both ops always evaluate to the same value. But in the second case, tf.get_default_graph().get_operations() will reveal a new operation in the list called Identity.

    sess = tf.InteractiveSession()
x = tf.Variable(1.0)
sess.run(x.initializer)

ref = x
copy = tf.identity(x)
print(x.eval(), copy.eval(), ref.eval())  # 1.0 1.0 1.0

sess.run(x.assign(2.0))
print(x.eval(), copy.eval(), ref.eval())  # 2.0 2.0 2.0

print(tf.get_default_graph().get_operations())

    You may wonder, why would anyone want to introduce a new op when she can simply make an assignment. There cases when assignment doesn't work, but tf.identity does, exactly because it creates a new op, e.g. in control flow. See this question: How to add control dependency to Tensorflow op.