Sampling correlated gaussian random variables

Hi all,

Say I have a vector \mathbf{x} = (\mathbf{x_1},\mathbf{x_2},\mathbf{x_3}) (all three are vectors of some lengths themselves), and I have the full non-diagonal covariance C for \mathbf{x}. The log likelihood is then proportional to \mathbf{x}^T C^{-1} \mathbf{x}. If I want to sample the entire \mathbf{x}, then I can just use pm.MvNormal. However, if I already know \mathbf{x_3} from data and want to fix it. Is there a simple way to sample (\mathbf{x_1},\mathbf{x_2},\mathbf{x_3}) with x_3 fixed to what I already have?

(I don’t know if this is relevant, but the end goal I am trying to achieve is to compare some function of \mathbf{x_1},\mathbf{x_2} with observed values using HMC. I know how to do this if everything is uncorrelated, but not sure about the correlated case). Thank you for any suggestions in advance!


You need to express x1 and x2 conditioned on x3: Multivariate normal distribution - Wikipedia, which then gives you 2 expressions for mu and cov that you can plug into a pm.MvNormal for x1 and x2.

An easier approach is to model x3 as a linear combination of x1 and x2 (ie., adding a regression into your model), they are more or less equivalent.

1 Like