I was interested in using the Elliptical Slice sampler for a GP to compare performance against NUTS, but best I can tell it was implemented in 2017, moved to pymc-experimental in 2019, which was archived in 2021. I know it still exists in blackjax, but [being forced to work] on a windows architecture, I can’t install jax. Are there alternative places where the Elliptical Slice sampler is implemented these days?
Alternatively, what other samplers are implemented that are worth considering for GPs?
From memory I think that elliptical slice sampler that was in PyMC required the other variables like lengthscale to be fixed. I think the hard part is then mixing the slice sampler for the GP with NUTS for the other variables. I think doing a NUTS-within-Gibbs then would require you to redo the NUTS tuning phase at each step, which would be pretty slow. I have kind of low confidence in this answer though, sorry! Hope it’s helpful anyway. EDIT: better info here.
Haven’t tried blackjax’s ESS implementation but am curious. But if your on windows then that wont work. Maybe docker or can you set up WSL? You could also checkout INLA.
NUTS does work well for GPs though, but since GPs are so flexible and can introduce funnels you get most of your speedups from carefully choosing priors. And if your problem is low dimensional, def try HSGPs!
Thanks @bwengals, definitely looking forward to the talk on HSGPs. Definitely want to spend some time with INLA, I know a couple people using it for similar problems to good success. Always makes me sad there’s no python version, but I can muscle up and use R just this once