PyMC3 large MvNormal prior

Question

I want to specify large multivariate normal distribution as a prior in PyMC3. The precision matrix of this distribution has determinant numerically equal to zero. It seems this is a problem for PyMC3. Any suggestions? I only need to maximize the posterior, which can be done regardless of the value of the determinant.

Answer 1

pymc3 gets the determinant by computing the cholesky decomposition. It also does this on a log-scale, so this really shouldn't underflow. It is possible that the matrix is ill conditioned and the cholesky decomposition fails however. In this case you could add a small diagonal to your matrix.

If you are sure that you want to work with an ill conditioned matrix, you could write your own version of pm.MvNormal, that doesn't include the det. Something along the lines of this:

class MvNormalNoDet(pm.Continuous):
    def __init__(self, mu, tau, *args, **kwargs):
        self._mu = tt.as_tensor_variable(mu)
        self._tau = tt.as_tensor_variable(tau)
        self.mean = self.median = self.mode = self._mu
        super().__init__(*args, **kwargs)

    def logp(self, value):
        diff = value - self._mu
        return -0.5 * (diff * tt.dot(self._tau, diff)).sum(axis=-1)