Zero and NaN MCMC samples... floating point precision error?

Has anyone experienced sudden zero and NaN MCMC samples when sampling from the posterior predictive distribution?

I am using a lognornal likelihood and a few regression coefficients with initially uniform priors. After an observation, the new priors for the regression coefficients are take from the posterior (using DensityDist and KDE). MCMC samples are used from the posterior predictive distribution (using the posterior regression coefficients and an independently obtained estimate of the next observation) to provide an estimate of the random variable of interest.

However, after about 30 or so iterations of this setup, all of the MCMC samples from the posterior predictive distribution are either zero or NaN. I am new to Bayesian modeling and PyMC3 but I have heard of floating point errors and I am wondering if anyone knows if this is related?

https://www.johndcook.com/blog/2012/07/26/avoiding-underflow-in-bayesian-computations/

I am clueless as to how to stat to work around this in PyMC3 and I would really appreciate anyone’s input on how to start fixing this issue.

Some verbose logging showing the MCMC Posterior predictive distribution samples of successful predictions followed by the sudden NaN and zero samples are shown here:

The model code is here:

It could be that some parameters drop out of the support of the input domain. e.g., a negative sigma value for the standard deviation. You should check the trace.
It is quite difficult to tell without the data. To better debug it, could you please isolate the input, output (trace), and model of the failed case?

Thanks! Turns out I was taking the log of something that at times was zero, which results in NaN.

2 Likes