Why TensorFlow eager execution API give a wrong answer for this function?

Question

I had this function to get its differentiation value.

def dp1_f1(x):
return 64*x*(1-x)*(math.pow((1-2*x),2) )*math.pow((1-8*x+8*x*x), 2)

I want to get dy/dx value. I can get this value by numeric method just as below:

def dp_numeric_diff(x):
    delta_x = 0.0001
    return (dp1_f1(x+delta_x)-dp1_f1(x))/delta_x

I use TensorFlow eager execution API to calculate this value:

def dp_ad_tfe(x):
    tf.enable_eager_execution()
    tfe = tf.contrib.eager
    grad_lx = tfe.gradients_function(dp1_f1)
    x = 3.0
    y = dp1_f1(x)
    rst = grad_lx(x)
    return y, rst[0]

I call this function with code below:

numeric_diff = dp_numeric_diff(x)
print('Numeric method：{0}'.format(numeric_diff))
v, d = dp_ad_tfe(x)
print('TFE：{0}'.format(d))

It will display something like this:

Numeric method：-75290405.66440672
TFE：-19208000.0

I am sure that the numeric method is right. What's wrong with my TensorFlow eager execution code? By the way the same TensorFlow eager execution code can get correct answer for simple function like x^2.

Answer 1

TensorFlow's automatic differentiation APIs can only differentiate through compositions of TensorFlow operations, not through functions like math.pow() or other libraries. If you replace math.pow() with tf.pow() it should work out just fine.

Something like:

import tensorflow as tf
tf.enable_eager_execution()

def dp1_f1(x):
    return 64*x*(1-x)*(tf.pow((1-2*x),2) )*tf.pow((1-8*x+8*x*x), 2)

def dp_numeric_diff(x):
    delta_x = 0.0001
    return (dp1_f1(x+delta_x)-dp1_f1(x))/delta_x

grad = tf.contrib.eager.gradients_function(dp1_f1)

print(dp_numeric_diff(3.0).numpy()) # Prints -75300000.0
print(grad(3.0)[0].numpy())         # Prints -75279680.0

Hope that helps.

(Seems this was also asked on GitHub)