Pytorch torch.linalg.qr is differentiable?

Question

I have a neural network that involves the calculation of the QR decomposition of the input matrix X. Such a matrix is rectangular and it has maximal rank.

My question is if this operation still allows to make the gradients propagate backward during the training or if there can be some issue.

Answer 1

You are right in your suspicions. It is not always differentiable. Depending on the mode parameter, there are three different cases you need to consider.

From the documentation;

The parameter mode chooses between the full and reduced QR decomposition. If A has shape (*, m, n), denoting k = min(m, n)

mode= ‘reduced’ (default): Returns (Q, R) of shapes (, m, k), (, k, n) respectively. It is always differentiable.

mode= ‘complete’: Returns (Q, R) of shapes (, m, m), (, m, n) respectively. It is differentiable for m <= n.

mode= ‘r’: Computes only the reduced R. Returns (Q, R) with Q empty and R of shape (*, k, n). It is never differentiable.