Modelling multivariate distributions with a sparse covariance structure

[Sina_Mansour_L]

Thanks for your helpful suggestions. Here is some extra information regarding your comments:

@ckrapu

Do you have any known grouping structure to this data? For example, if you get 2000 dimensions from 10 variables X 40 timesteps X 5 spatial locations, you could use a Kronecker structured Normal model to represent the 2000-dimensional covariance in terms of the Kronecker product of 10x10, 40x40 and 5x5 matrices.

Not really, these 2000 kernels are in fact a dimensionality-reduced subspace of a higher dimensional feature set (90K feature dimensions that are not independent).

@ckrapu

Is a low rank assumption for the covariance okay? Or do you want to get actual zeros (or close to zero) values for the posterior off-diagonal elements?

Interesting question, I don’t think I want to enforce zeros, I just thought that would simplify the complexity.

Is there a way to test if a low-rank covariance structure exists in the data?

I have extracted some plots to visualize the sparsity of the covariance.

Here is the 2000x2000 correlation matrix across kernels (absolute values of Pearson’s correlation are plotted):

and this is the histogram of the upper triangle (notice the y-axis is logarithmic):

To me, these indicate that the majority of the correlations are close to zero and that’s why I expected a sparse model would help.

Is there a way to test if the low-rank assumption holds?