Modelling multivariate distributions with a sparse covariance structure

Questions v5
2

There’s a few options here:

  1. Do you have any known grouping structure to this data? For example, if you get 2000 dimensions from 10 variables X 40 timesteps X 5 spatial locations, you could use a Kronecker structured Normal model to represent the 2000-dimensional covariance in terms of the Kronecker product of 10x10, 40x40 and 5x5 matrices.

  2. Is a low rank assumption for the covariance okay? Or do you want to get actual zeros (or close to zero) values for the posterior off-diagonal elements? If the former, you can try something like this:

# m is the rank of the resulting covariance matrix
with pm.Model() as model:
    sd = pm.HalfCauchy('sd', 2.5)
    low_rank = pm.Normal('low_rank', mu=0, sd=sd, shape=(p, m))
    cov = pm.Deterministic('cov', pm.math.dot(low_rank, low_rank.T))

    # Sample from multivariate normal
    Y = pm.MvNormal('Y', mu=np.zeros(p), cov=cov, shape=(n, p))
    trace_low_rank = pm.sample(1000)

Here’s a Colab notebook with a comparison between LKJ and low rank covariance priors.

1 Like