Hi y’all! Long time lurker, first time poster. I recently read @aloctavodia 's book on Bayesian Analysis with Python and enthusiastically fitted a Gamma BART model last Friday. Today when I came back to the model it won’t even make it one step into sampling and fails with “OverflowError: Python int too large to convert to C long pymc”. From the snooping that I have done so far it seems like it could maybe be an issue with the leapfrog step size? I did not change anything about the model or the data and I did set a random seed so I am pulling my hair out as to what might be going on here. Has anyone seen this error before? It seems super hard to track. Thanks!
Could you post the model code you ran, along with the full error traceback you got?
Unfortunately I can’t post the full model but it is a model with a BART variable that I pass into a pm.Gamma() using its mu and sigma. The point of failure happens at pm.sample() and the key points in the stack trace are:
strace.record(draw.point, draw.stats)
for data, vars in zip(self._stats, sampler_stats):
for key, val in vars.items():
data[key][draw_idx] = val
elif self._stats is not None:
raise ValueError(“Expected sampler_stats”)
OverflowError: Python int too large to convert to C long
The same model with doing only pm.sample_prior_predictive() instead of pm.sample() doesn’t cause any issues and when I inspect the prior samples there are no values that are unusual.
Can you show the full original traceback?
Yup:
In sample(draws, tune, chains, cores, random_seed, progressbar, progressbar_theme, step, var_names, nuts_sampler, initvals, init, jitter_max_retries, n_init, trace, discard_tuned_samples, compute_convergence_checks, keep_warning_stat, return_inferencedata, idata_kwargs, nuts_sampler_kwargs, callback, mp_ctx, blas_cores, model, compile_kwargs, **kwargs) 926 _print_step_hierarchy(step) 927 try: → 928 _mp_sample(**sample_args, **parallel_args) 929 except pickle.PickleError: 930 _log.warning(“Could not pickle model, sampling singlethreaded.”), in _mp_sample(draws, tune, step, chains, cores, rngs, start, progressbar, progressbar_theme, traces, model, callback, blas_cores, mp_ctx, **kwargs) 1405 strace = traces[draw.chain] 1406 if not zarr_recording: 1407 # Zarr recording happens in each process → 1408 strace.record(draw.point, draw.stats) 1409 log_warning_stats(draw.stats) 1411 if callback is not None: in NDArray.record(self, point, sampler_stats) 114 for data, vars in zip(self._stats, sampler_stats): 115 for key, val in vars.items(): → 116 data[key][draw_idx] = val 117 elif self._stats is not None: 118 raise ValueError(“Expected sampler_stats”) OverflowError: Python int too large to convert to C long
Yes this is very weird one. Did you see any of these already on pymc-bart @aloctavodia @gstechschulte ?
How did you install PyMC and pymc-bart @krishasenbein , which package versions do you have installed, and which OS (Mac, Linux or Windows) – maybe that could be a reason @jessegrabowski @ricardoV94 ?
Sorry for the late reply. @krishasenbein This should be fixed now. Can you try installing pymc-bart from GitHub `pip install git+https://github.com/pymc-devs/pymc-bart.git` (a new release will be available soon).
Hi @aloctavodia ! Just tried the same model that was giving me this issue before and it is running fine now, no more error. Thank you for the fix! Out of curiosity, what was the issue? 
It was discovered here: OverflowError: Python int too large to convert to C long
Thanks! I am seeing issues now with plot_convergence() which is not outputting anything anymore, and compute_variable_importance(), which is now giving me an issue about “sample_stats”:
`File ~/.venv/lib64/python3.11/site-packages/pymc_bart/utils.py:887, in compute_variable_importance(idata, bartrv, X, model, method, fixed, samples, random_seed)
882 predicted_all = _sample_posterior(
883 all_trees, X=X, rng=rng, size=samples, excluded=None, shape=shape
884 )
886 if method in [“VI”, “backward_VI”]:
→ 887 vi_xarray = idata[“sample_stats”][“variable_inclusion”]
888 if vi_xarray.variable_inclusion_dim_0.size > 1:
889 if model is None:
File ~/.venv/lib64/python3.11/site-packages/arviz/data/inference_data.py:271, in InferenceData.getitem(self, key)
269 “”“Get item by key.”“”
270 if key not in self._groups_all:
→ 271 raise KeyError(key)
272 return getattr(self, key)
KeyError: ‘sample_stats’`
Was there any change that could have impacted this?
`plot_convergence` was deprecated in favor of arviz_plots.plot_convergence_dist — arviz-plots dev documentation . You should get a warning message.
The error is telling that your idata object does not have a `sample_stats` group, which is weird. Can you double-check?