raise ValueError No gradients provided for any variables -Custom loss function

Question

I'm writing a custom loss function for a neural network with keras back-end, to reduce attitude error.

def LossQuat(y_true, y_pred):
        a, b = y_true.get_shape()
        error  = np.zeros([a,1])
        for i in range(a):
            w0,x0,y0,z0 = y_true[i,:]
            w1,x1,y1,z1 = y_pred[i,:]/tf.norm(y_pred, ord='euclidean', axis=None, keepdims=None, name=None)
            w = w0 * w1 - x0 * x1 - y0 * y1 - z0 * z1
            error[i,] = tf.square(2*tf.math.acos(w))
        err = tf.reduce_sum(error)
        return tf.reduce_mean(err) 

The model is:

model = keras.models.Sequential()
model.add(keras.layers.Dense(100, input_dim=1, activation='sigmoid'))
model.add(keras.layers.Dense((4), activation='linear'))

model.compile(loss=LossQuat, optimizer = tf.keras.optimizers.Adam(lr=1e-5),run_eagerly=True)

# training
batch_size = 32
epochs = 1000
model.fit(x_train, quat_pitch, batch_size=batch_size, epochs=epochs, verbose=1)

But I got this error:

raise ValueError(f"No gradients provided for any variable: {variable}. "

ValueError: No gradients provided for any variable: (['dense/kernel:0', 'dense/bias:0', 'dense_1/kernel:0', 'dense_1/bias:0'],). Provided grads_and_vars is ((None, <tf.Variable 'dense/kernel:0' shape=(1, 10) dtype=float32, numpy= array([[ 0.027, 0.718, 0.436, 0.588, 0.597, -0.712, 0.038, 0.629, 0.305, 0.463]], dtype=float32)>), (None, <tf.Variable 'dense/bias:0' shape=(10,) dtype=float32, numpy=array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0.], dtype=float32)>), (None, <tf.Variable 'dense_1/kernel:0' shape=(10, 4) dtype=float32, numpy= array([[ 0.449, 0.503, -0.456, 0.521], [-0.365, 0.423, 0.55 , 0.032], [-0.311, 0.348, -0.056, 0.174], [ 0.521, 0.498, -0.131, -0.507], [-0.107, 0.321, 0.638, 0.117], [ 0.248, 0.416, -0.259, -0.273], [ 0.121, 0.137, -0.575, 0.094], [ 0.41 , -0.565, -0.394, -0.239], [-0.531, -0.056, 0.13 , 0.201], [ 0.225, -0.122, 0.556, -0.266]], dtype=float32)>), (None, <tf.Variable 'dense_1/bias:0' shape=(4,) dtype=float32, numpy=array([0., 0., 0., 0.], dtype=float32)>)).

What is my mistake?

I tried to use Keras Loss Functions such as MSE. When I used them the model has been trained without any problem.

Also, for the custom loss, I tried different batch size, for all of them the problem is exist.

DATA SHAPE

x_train.shape = (304414,)
y_train.shape = (304414,4)

MATH:

The main formula is as follows:

quat_predict * inverse(quat_ref) = [w ,x, y, z]

The error:

error = 2*arccos(w)

Answer 1

Tensorflow cannot track gradients through object assignments like your code with errors. This creates a copy of a value, and thus the gradient is not defined. Instead you should put things into a list, or vectorise the whole thing.

def LossQuat(y_true, y_pred):
        a, b = y_true.get_shape()
        error = []
        for i in range(a):
            w0,x0,y0,z0 = y_true[i,:]
            w1,x1,y1,z1 = y_pred[i,:]/tf.norm(y_pred, ord='euclidean', axis=None, keepdims=None, name=None)
            w = w0 * w1 - x0 * x1 - y0 * y1 - z0 * z1
            error.append(tf.square(2*tf.math.acos(w)))
        err = tf.reduce_sum(error)
        return tf.reduce_mean(err) 

Note that now the list is composed of tf.Tensor object that has a functional dependency on your predictions. Before the output would be a numpy float, with a dependency lost.