I’m getting a lot of these errors when saving a trace:
/work/05204/rpg/jupyter_packages/envs/xplan-experiment-analysis/lib/python3.9/site-packages/arviz/stats/diagnostics.py:642: RuntimeWarning: invalid value encountered in double_scalars
between_chain_variance / within_chain_variance + num_samples - 1) / (num_samples)
any idea what could trigger this? Should it be an error instead of a warning?
Has everything converged? Something similar happened to me once and I had some chain with a constant value for all draws.
Also, from the variable names in the line shown, this looks like the code of rhat, it would be great if you could run rhat for some subsets of the data and see for which variables does this happen or if it happens for all of them, maybe it has to do with dtypes?
I’ll check that. I note that this model has an enormous number of deterministic variables that combine together features from a number of different vectors based on the value of predictors. There’s probably a question about model coding about how to remove those…
Meanwhile, if I use the var_names and filter_vars arguments to filter out those deterministic variables, all of these warnings go away.
@OriolAbril , may I ask, from your experience, what’s preventing the chain from converging, or rather, from moving forward at all? I am running into a similar type of issue, with many variables involved, since it is a simple problem solved simultaneously for nearly 500 locations (because I’d like to sample a common, global driver on top of local parameters). The chains converges well when the problem is posed for each location individually (which is not a very different situation from the global problem, so I’d expect solutions exist), or when I increase the observation variance by a rather large factor. Formulated this way, clearly it has to do with the low likelihood of a single draw, but I thought a MCMC was good at solving just that… Any workaround ? Play with the acceptance probability? For now simply increasing the tuning steps from 1000 to 100000 does not improve the situation.