Does auto-differentiation in tensorflow work when combining activations from multiple nets into one objective?

Question

I am new to tensorflow and trying to figure out if the auto-differentiation feature in tensorflow will solve my problem. So I have two nets , where each net outputs a latent vector. So let's say my net A outputs latent vector -La(Hxr) - where (H,r) represent the dimensions of the output latent vector La. Similarly net B outputs Lb(Wxr) . So my objective functions takes both the two latents as input and combines them like (La.Lb') where (.) is dot product and (') represents the transpose . I will be optimizing this objective function using cross-entropy.

Now my question is will the tensor-flow auto-diff be able to calculate the gradients correctly and back propagate? It's not a straight forward case here. The net A should only be updated from the gradients w.r.t. La and net B should only be updated with gradients calculated w.r.t Lb. So is tensorflow smart enough to figure out that? And is there a way to validate this?

Thanks!

Answer 1

TensorFlow supports auto-differentiation of whatever kind of computational graph you can define using it. I have used TensorFlow to combine the predictions from multiple Nets to compute the loss using different loss functions. So, tensorflow is smart enough to figure out this, it will compute the gradients correctly and back propagate it.