Sampling correlated gaussian random variables

Questions version agnostic
1

Hi all,

Say I have a vector \mathbf{x} = (\mathbf{x_1},\mathbf{x_2},\mathbf{x_3}) (all three are vectors of some lengths themselves), and I have the full non-diagonal covariance C for \mathbf{x}. The log likelihood is then proportional to \mathbf{x}^T C^{-1} \mathbf{x}. If I want to sample the entire \mathbf{x}, then I can just use pm.MvNormal. However, if I already know \mathbf{x_3} from data and want to fix it. Is there a simple way to sample (\mathbf{x_1},\mathbf{x_2},\mathbf{x_3}) with x_3 fixed to what I already have?

(I don’t know if this is relevant, but the end goal I am trying to achieve is to compare some function of \mathbf{x_1},\mathbf{x_2} with observed values using HMC. I know how to do this if everything is uncorrelated, but not sure about the correlated case). Thank you for any suggestions in advance!

Best,
Alan

2

You need to express x1 and x2 conditioned on x3: Multivariate normal distribution - Wikipedia, which then gives you 2 expressions for mu and cov that you can plug into a pm.MvNormal for x1 and x2.

An easier approach is to model x3 as a linear combination of x1 and x2 (ie., adding a regression into your model), they are more or less equivalent.

1 Like