I am trying to run a Raftery diagnostics on my pymc sampler as suggested here: 7. Model checking and diagnostics — PyMC 2.3.6 documentation
This is to check the number of samples needed for convergence for accurately estimating an interval like 0.025 quantile of the posterior.
However, my latest pymc install doesn’t have the raftery_lewis() function. I keep getting a “module ‘pymc’ has no attribute ‘raftery_lewis’” error. I can’t seem to find the latest name for this function in pymc>=5. Does anyone know what the function’s name has changed to or how to call the function in later pymc versions?
Maybe @aloctavodia will know. My guess is that method is no longer seen as very useful compared to rhat and other modern approaches.
In either case diagnostic methods are implemented in the arviz package these days, not in PyMC
1 Like
Hi @anamoly,
As Ricardo already mentioned, diagnostics are implemented in ArviZ, not PyMC. The stable version of arviz is here ArviZ: Exploratory analysis of Bayesian models — ArviZ 0.21.0 documentation. We tried to keep ArviZ diagnostics updated to the most modern, reliable, and useful diagnostics and practices. For instance, thinning is no longer a recommended practice (with a few exceptions).
You can compute the mcse for a quantile with something like az.mcse(idata, method="quantile", prob=0.25). That will give you the error introduced by the sampling method.
As a rule of thumb, for a nicely behaved model and using NUTS sampler (with the defaults 4 chains of 1000 draws each), we should expect 2 significant digit accuracy for the posterior mean and 1 or 2 for the tails. But you can use mcse to check if that’s the case for your particular problem
2 Likes