Gradient are None for sliced input in TensorFlow

Question

Following is my code

import tensorflow as tf
import numpy as np
def forward(x):
  z = tf.Variable(tf.zeros_like(x), trainable=False)
  s = tf.shape(x)[0]
  for i in range(s):
    z[i].assign(x[i]**i)
  return z

a = tf.Variable(np.ones([5])*3)

with tf.GradientTape() as tape:
  b = forward(a)
grad = tape.gradient(b, a)

I have a input which I have to slice and then calculate the output. Upon which, I need to calculate gradients. However, the output of above code is None.

How can I obtain gradients? Is there any way in which I can slice the input to get the gradients.

P.S. I have to only use EagerExecution. No graph mode.

Answer 1

When using gradientTape, it is helpful if you think as a function. Let's supposed that your cost function is y = x ** 2. It is possible to calculate the gradient of y (your function) with respect to x (your variable).

In your code, you don't have a function to calculate the gradient. You were trying to calculate a gradient against variables and that does not work.

I have done a small change. Check the variable cost in the code below

import tensorflow as tf
import numpy as np
def forward(x):
  cost = []  
  z = tf.Variable(tf.zeros_like(x), trainable=False)
  s = tf.shape(x)[0]
  for i in range(s):
    z[i].assign(x[i]**i)
    cost.append(x[i]**i)
  return cost

a = tf.Variable(np.ones([5])*3)

with tf.GradientTape() as tape:
  b = forward(a)
grad = tape.gradient(b, a)
print(grad)