Incremental updates to the hyper-priors in hierarchical model

Hi there,

I currently have a Bayesian hierarchical model working, and I want to be able to perform incremental learning as new data comes in.

I have seen ‘Updating Priors’ example in the PyMC3 documentation (Updating priors — PyMC3 3.11.4 documentation), which uses the posteriors to update the priors. However, since I am working with a hierarhcial model I would instead need to update the hyper-priors using the posteriors.

For example, suppose that for each coefficient’s prior is modelled using a Normal distribution, with a Normal distribution for the location hyper-prior and a strictly positive distribution for the dispersion hyper-prior. How would I then use my Normally distributed posteriors to automatically setup these two hyper-prior distributions?

Any suggestions and/ or thoughts would be much appreciated.

Hyper-priors are also part of the unknown parameter in your model, thus after inference you get posterior distribution for them as well (in the format of MCMC samples).

Hi @junpenglao, thank you very much for getting back to me.

My apologies here, you are correct, but in my attempt to keep the post simple I neglected some details and probably confused you. Instead of incremental learning of the same model, I am actually running an up-stream pooled model and then using that to kick start a different down-stream hierarchical model. Therefore I would need to use the posteriors from the up-stream pooled model, to set the hyper-priors of the down-stream hierarchical model. Do you know whether this is possible or how I should approach it? Any edvice from the experts would be much appreciated.

Alternatively, I have thought about implementing a 3-tiered hierarchical model, but there is almost no discussions on this (and one post I read actually discourgaed it).

Many thanks in advance!

Depending on the set up and the data, without more information it is hard to say which approach is better. I think it is fine to fit 2 models, and for setting up the 2nd model you can use the posterior to set up the hyperprior (approximating it with a Normal is a good start).
Since you are fitting first an up-stream model then a down-stream hierarchical model, I am assuming that you can aggregate the data for up-stream model. In this case, a 3 level model is also fine - you can construct the 3 level hierarchical model and assign observed to the intermedia. Otherwise by choosing a good prior you should be able to fit the model without problem.

Thank you for getting back to me again, and sorry that I can’t be more specific.

Do you have any links, resources, or example implemntations that of the 3 level hierarchical model I’d be interested to see the setup.

Thanks again.