When I use the text backend, I get an error about it not supporting sampler stats: import numpy as np import matplotlib.pyplot as plt import seaborn as sb import pandas as pd import pymc3 as pm from pymc3.backends import Text import os TRACE_DIR = os.path.join('your_path_here', 'data', 'test_trac…

Yep, text backend does not support sampler stats. You can try pickle the trace [image] Get nan or inf from model.logp (model.test point) is an attestation of incorrectly configured model? Questions Haven’t tried this, but it should work (give it a shot with very few sample…

It seems like the NUTS sampler automatically calls .get_sampler_stats, so I think this means it’s impossible to use the text backend with NUTS.

That is a legit bug, then – I can try to fix that. Out of curiosity, is there a reason you’re using the Text backend? It might be time to deprecate these, since I’m not sure of a use case that is not better covered by the standard NDAarray, if we add a .save and .load method to it. I think the co…

I’m using the text backend because I’m running a largish model in a low-memory environment - an EC2 instance with only 3GB of RAM. I’m not sure what the implications are of adding .save and .load methods to NDAarray are, but as long as there’s a way to run inference without storing the traces in m…

Thanks! I’ll keep that in mind. Back of the envelope, 3GB is ~750M float32's, but I suppose a 1M samples of 100 variables isn’t ridiculous, along with some other overhead. Frankly, I’m impressed you could install numpy/scipy on a machine of that size – I guess python wheels help?

Your back of the envelope math got me curious, so I went back and looked at my logs, and it looks like I had used text from the get-go - I was assuming that I would have memory issues if I didn’t - and that I was hitting memory errors here: ris = trace.get_values('ris_level_offset', burn=1000, …

You could also use the hdf5 backend, that should support sampler stats, too. The backends aren’t tested particularly well however (as you have seen yourself :frowning: )

It looks like you will get a memory error when you load the trace anyways, right? Something like this might require custom code on your part - it seems like the text backend is just storing things as a csv, so probably grabbing the file and using something like the standard library’s csv.DictReader …

This will at least get rid of the error – I spent an hour trying to implement sampler stats, but it was a little bit non-trivial, and I would rather not spend too much time on a feature that could easily disappear in a few months. https://github.com/pymc-devs/pymc3/pull/2431

Using text backend raises sampler stats error

Questions

junpenglao July 17, 2017, 6:15am 2

Yep, text backend does not support sampler stats. You can try pickle the trace

Topic		Replies	Views
Backends don't work with simple setup when sample sequentially Questions	1	1002	February 13, 2018
Typical workflow in PyMC3 Questions	2	1043	January 9, 2018
Loaded saved Text Trace does not show any summary or traceplot Questions	6	1809	May 29, 2018
Access sampler stats and saving burn-in Questions	1	622	January 25, 2019
Resuming sampling from a previous trace v5	12	1882	September 1, 2023

Using text backend raises sampler stats error

Related topics