Non-centered multivariate normal

Questions
1

I want to build a hierarchical model with let’s say 5 submodels and let’s say 5 stochastic RVs per submodel. I want to try a multivariate normal prior on all of 25 lower-level RVs, but I’m worried about funneling, so I want to use a non-centered parameterization. Has anyone written elegant PyMC3 code for this (a noncentered multivariate normal hierarchical model) already? It would be great not to reinvent the wheel!

2

Hi Sam,
Most of the models in the updated radon NB are non-centered, and there is a new MvNormal one :tada:
So I think you’ll find it useful :wink:

2 Likes
3

Thanks!

Did you write/update that notebook? I ask because I can’t see intuitively how that non-centered parameterization solves the funneling problem (in my use-case, I’m still getting divergent samples, centered or non-centered).

4

Yep, I wrote the updates of the different models – the ArviZ magic was done by the great @OriolAbril.

This can be due to a wide number of things, unfortunately. I’d check that priors are in line with your domain knowledge and regularizing enough (very important in hierarchial models), and if there is any strong correlations between parameters’ posteriors.
Also, to make things even more interesting (:stuck_out_tongue_winking_eye:), if the posterior of the population sigma (the one which induces shrinkage of parameters) is far enough from zero, the centered parametrization will usually be better suited.
Hope this helps :vulcan_salute:

2 Likes
5

Just wanted to say thanks for the help! My non-centered multivariate hierarchical model seems to be working well. I did have to play with target_accept quite a bit, though.

1 Like
6

Thanks for the feedback, really glad to know when a resource is useful!

7 
# Expected value per county:
theta = (
        mu_alpha_beta[0]
        + alpha_beta_county[county_idx, 0]
        + (mu_alpha_beta[1] + alpha_beta_county[county_idx, 1]) * floor_idx
    )

Thank you so much for the radon lesson! I have a question regarding MvNormal non-centered parameterization, how would you be able implement the theta when if you want to add in another variable, for example, room. Instead of bivariate, I want to implement a tri-variate, making it a 3x3 chol.

8

Hi @LawrenceX , and sorry for the late reply.
You should be able to just add another dimension to your MvNormal. Does that make sense?