I previously asked about improving model sampling speed fitting about 14K observations to about 200 groups. I am fitting a hierarchical model with Negative Binomial(alpha, mu) distributed likelihood (observations underneath level_2). The priors for the parameters alpha and mu (level_2) are Gamma(hy…

The first thing I would try is changing the Gamma to a HalfNormal and use a non-centered parameterization. Do you observed any speed up of NUTS before and after the tuning sample?

By tuning sample, do you mean the initialization sample for ADVI? If so, for ADVI, it was about 30 it/s for the initialization and 1.5 s/it for the sampling from NUTS. [05 AM] Is there any literature on how to use a non-centered parameterization?

By tuning I meant the speed of NUTS before and after 500 samples (ie, the default number of tuning). I am asking because if there is a speed difference, it indicates the initialization from ADVI is not good, PyMC3 ends up tuning the mass matrix in NUTS anyway, which sometimes could be quite long. T…

Hmm, I do recall that the first 500 samples appeared faster under the NUTS sampling. Should I specify more initial samples during ADVI? Thank you for the blog posts on parameterization; I’ll check those out and see if they help.

If you have the first 500 samples faster, it could be the case that the sampler fall into some kind of funnel. You should run multiple chain (e.g., njobs=4) and check the trace. Did you see any divergent warning?

I did not see any divergent warning. I’ll retry with njobs=4, but doesn’t that default to the number of cores automatically which would be 4?

Nope, you need to specify the number of chains. :slightly_smiling_face:

FYI, have a look at this post for speed related diagnostic.

Ok got it. Thank you for your advice. I’ll reply after trying the reparametrization and checking divergence.

Hello, none of the links posted demonstrate how to reparametrize distributions outside of the location-scale family. Is it still possible to reparametrize the half-normal to achieve the same thing?

Improving speed with large dimensional problem

Questions

junpenglao September 27, 2017, 5:32pm 13

Oops, here is the link Frequently Asked Questions

Topic		Replies	Views
Improving model convergence and sampling speed Questions	7	6391	April 12, 2019
Slow speed of NUTS in Hierarchical model Questions	14	2253	October 20, 2020
Hierarchical Model - Slow Sampling Questions	4	1232	March 26, 2020
Feasibility of large hierarchical multiple regression v5 modeling	4	336	October 24, 2023
NUTS sampler slowing down on a hierarchical model Questions	6	1015	February 28, 2021

Improving speed with large dimensional problem

Related topics