R_hat statistics not 1, but no divergences

How bad is it that the r_hat statistics of my model coefficients are not exactly 1.00?

I have some that are around values of 1.01-1.03.

What is strange is that I have no model divergences so should these r_hat values still be alarming?

I have a relatively complicated model and I am already training it with large burn-in period and a high target acceptance rate. I have also tried other things, such as better priors and centred vs. non-centred approaches. I know the r_hats should ideally be 1.00, but I am working with a very noisey dataset and I am starting to wondering at what point of ‘not perfect’ is acceptable.

Thanks!

As a practical guide, I’ve seen thresholds for \hat{R} in different disciplines that say anything under 1.2, 1.1, and so on are all okay. The truth of it is that the usage of this diagnostic is not intended to be a test statistic for whether or not you’ve got a good posterior estimate, but rather instead to use as an alarm when it’s way off. I wouldn’t worry too much about those values being different than 1.00, especially if you have small numbers of samples and highly autocorrelated chains due to model complexity.

In the past, I’ve seen models with ~100,000 parameters fitted using NUTS on the GPU, and even if everything is done very well and the posterior geometry is nearly ideal, simply having that many random variables may give you a few \hat{R} values outside of the ideal range if only because that statistic is itself a random variable generated from a stochastic process.

2 Likes

As always, I like to point to Aki’s keynote from 2020’s PyMCon. He talks about specific diagnostics, but I think the more important lesson is that one can always subject simple (sometimes opaque) diagnostics (e.g., \hat{R}, ESS, # of divergences) to further investigation. So there’s no need to rely on simple heuristics/thresholds (e.g., \hat{R} should be no more than X).

2 Likes