Samplers for GPs? (alternatively, where should I find the elliptical slice sampler without Jax?)

Questions v5
1

I was interested in using the Elliptical Slice sampler for a GP to compare performance against NUTS, but best I can tell it was implemented in 2017, moved to pymc-experimental in 2019, which was archived in 2021. I know it still exists in blackjax, but [being forced to work] on a windows architecture, I can’t install jax. Are there alternative places where the Elliptical Slice sampler is implemented these days?

Alternatively, what other samplers are implemented that are worth considering for GPs?

1 Like
2

If you care about scalability of GPs, this is your best bet: Add HSGP Latent GP approximation by bwengals · Pull Request #6458 · pymc-devs/pymc · GitHub There’s also an upcoming pymcon webinar on this topic: PyMCon Events · PyMCon Web Series

If you really want the sampler, easiest would be to dig it up from github and try to run it locally.

2 Likes
3

Thanks @twiecki, was actually checking out different samplers for scaling GPs exactly in anticipation for that talk.

2 Likes
4

From memory I think that elliptical slice sampler that was in PyMC required the other variables like lengthscale to be fixed. I think the hard part is then mixing the slice sampler for the GP with NUTS for the other variables. I think doing a NUTS-within-Gibbs then would require you to redo the NUTS tuning phase at each step, which would be pretty slow. I have kind of low confidence in this answer though, sorry! Hope it’s helpful anyway. EDIT: better info here.

Haven’t tried blackjax’s ESS implementation but am curious. But if your on windows then that wont work. Maybe docker or can you set up WSL? You could also checkout INLA.

NUTS does work well for GPs though, but since GPs are so flexible and can introduce funnels you get most of your speedups from carefully choosing priors. And if your problem is low dimensional, def try HSGPs!

5

Thanks @bwengals, definitely looking forward to the talk on HSGPs. Definitely want to spend some time with INLA, I know a couple people using it for similar problems to good success. Always makes me sad there’s no python version, but I can muscle up and use R just this once :slight_smile:

1 Like