Hi!
I’m working on implementing the PyMC NUTS sampler into the MontePython code (GitHub - brinckmann/montepython_public: Public repository for the Monte Python Code) to use it with complex cosmological datasets.
MontePython will handle the log-likelihood and gradient calculations, and this takes two types of variable parameters: “cosmo” (the actually important cosmological parameters which are computed externally) and “nuisance” (stuff like noise correction parameters within datasets which are much faster to calculate). The default MCMC method samples nuisance parameters at a higher rate (typically 2-4x) as they are fast to vary. Is there a way to define this sort of oversampling when feeding the parameters into PyMC?
You mean that multiple draws of the nuisance parameters are taken for every draw of the cosmo variables?
NUTS samples all the variables at the same time by default. You could assign the variables manually to two NUTS samplers, but you would need to subclass/ override the default behavior which is to perform one update each draw.
Does it really buy you anything in performance? You’ll likely have smaller effective sample size for the parameters you update slower.
Yes, I think that’s what it translates to in PyMC.
When cosmo variables are changed it requires running a complex external code to re-compute the cosmology whereas changing a nuisance parameter only affects the simple python functions that handle the partial likelihood of the relevant data, so in regular MCMC oversampling significantly improves convergence time. It’s probably less impactful in NUTS because of the overall improvement in efficiency but it would be a nice-to-have.
So you would need a step sampler that behaves exactly like the original one (not sure if you want to use NUTS or some other MCMC (Metropolis, Slice…) for the “slow-changing” variables), that returns the same value every n iterations, and only then proposes a new one.