Search code examples
pythontensorflowautodiff

How can I tell if a tf op has a gradient or not?

I am interested in using a SparseTensor in tensorflow, however, I often get

LookupError: No gradient defined for operation ...

Apparently gradient computation is not defined for many ops for sparse tensors. Are there any easy ways to check if an op has a gradient or not before actually writing and running my code?

Solution

  • There is a get_gradient_function function in tensorflow.python.framework.ops. It accepts an op and returns a corresponding gradient op. Example:

    import tensorflow as tf
from tensorflow.python.framework.ops import get_gradient_function

a = tf.add(1, 2, name="Add_these_numbers")
b = tf.multiply(a, 3, name='mult')

mult = tf.get_default_graph().get_operation_by_name('mult')
print(get_gradient_function(mult))  # <function _MulGrad at 0x7fa29950dc80>

tf.stop_gradient(a, name='stop')
stop = tf.get_default_graph().get_operation_by_name('stop')
print(get_gradient_function(stop))  # None