Strategies for validating Gaussian Process models

Take a look at this paper to see if LFO is what you are searching for, from the post it does look like a good fit. It is not as efficient as PSIS-LOO (which generally requires no refits) but it could give you leave future out cross-validation results with only a handful of refits (not sure how many though).

LFO is not yet implemented in ArviZ, but the infrastructure to allow ArviZ to call PyMC3 and refit the model on subsets of the data is: see this other discourse answer and links there. I don’t have time right now to add LFO to ArviZ, but if you are interested in working on it I would gladly help.

2 Likes