The MarginalSparse class represents a GP implementation where you’re using inducing points, and you’ve integrated out the GP f (which I think you need explicitly since it is your intercepts). I think I understand that the way you have it coded originally works, but that you’re looking for speedups. How big is K? I think trying to use a sparse covariance matrix will only help you if K is very large (and has many zeros), and your slowdown is due to repeatedly inverting K. If K is too big, I suspect you’ll also run into memory issues quickly. If K is reasonably sized and sampling what is slow, it could be because NUTS is having a hard time with strong correlations in your posterior. This seems pretty common for GPs because the lengthscale has a strong interaction with other parameters. I suspect that’s the case for this model. The best way to speed sampling up is to use more informative priors on the GP hyperparameters, lengthscale and eta. Instead of HalfCauchy, you could use Gamma’s.
1 Like