I have a question about how to improve model convergence and sampling speed. Currently, I am fitting a hierarchical model with Negative Binomial(alpha, mu) distributed likelihood. The priors for the parameters alpha and mu are Exponential(hyper_lam) and Gamma(hyper_mu, hyper_sd) distributed respect…

A few suggestion: 1, Avoid Uniform priors. You can have a look at the recommendation from Stan . Generally a weakly informative prior helps speed up the sampler, while providing much better regulation. 2, You dont need to specify y_pred = pm.NegativeBinomial( 'y_pred', mu=mu[cl…

Also, generally Metropolis is not suitable for high dimensional problem (although in your case I think it should perform OK). The speed in Metropolis samples is usually an illusion, as what counts is the effective samples. For this model, you can also try SMC, see an example here: http://docs.pymc.i…

From my personal experience, I usually find hierarchical models tend to sample very slowly with NUTS. A method I found that could help especially with large set of observations is to either approximate with ADVI and mini batch (which can speed up computation by 2 fold) and then sampling, if its goo…

Something to keep in mind is that ADVI with the default option (i.e., mean-field approximation) will underestimated the uncertainty of the parameters (much narrower posterior). You can have a look at this code of using different sampling on a mixed (hierarchical) model: https://github.com/junpengla…

@junpenglao - is there any literature about the amount by which ADVI underestimates, or the conditions under which underestimation is most severe for mean-field ADVI? Are other approximtions, such as fullrank ADVI also prone to this problem?

The original ADVI paper have some simulation study to show that (for hierarchical regression, at least). Full-rank ADVI also have the same problem but it is usually a bit better.

Improving model convergence and sampling speed

Questions

Topic		Replies	Views
Improving speed with large dimensional problem Questions	12	4106	September 27, 2017
Hierarchical Model - Slow Sampling Questions	4	1220	March 26, 2020
Slow speed of NUTS in Hierarchical model Questions	14	2236	October 20, 2020
NUTS sampler slowing down on a hierarchical model Questions	6	1004	February 28, 2021
My first (hierarchical) PyMC model is very slow v5	5	989	October 2, 2023

Improving model convergence and sampling speed

Related topics