Loaded saved Text Trace does not show any summary or traceplot

Questions
1

Hello,

I have a chain saved using the Text backend:

db = pm.backends.Text('save1')
trace = pm.sample(3000,step=step,start=start,trace=db)

I used this to load it within the context of my model:

with pm.Model() as model:
    test=pm.backends.text.load('save1')

and it worked, since test is recognized as variable : <MultiTrace: 1 chains, 3500 iterations, 0 variables> (not sure why it says 0 variables, since they are listed in the .csv file, but anyway).

Nevertheless, when I do this: pm.summary(test), nothing happens.
And when I try this: pm.traceplot(test), I get IndexError: index out of range in gridspec.py, a matplotlib function.

Any idea on how to get the summary and traceplot from there?

Thanks!

2

This sounds like a bug, but since there are plans to deprecate text backend maybe you should just pickle the trace instead.

3

I see, thanks.

I just read somewhere in the docs that Pickle should not be used for long-term project because the format itself changes often and gets deprecated faster, so I did not chose it.

But it’s definitely a better option? Because as for me, a CSV seems pretty simple yet robust…what will pickle bring instead?

4

I would say only pickling the trace as an intermedia step, and functions operating on the trace should work as intended (e.g., traceplot, sample_ppc etc). But you are right it is indeed better to save the trace as a CSV (for final report or paper), which you can do so by pm.trace_to_dataframe(trace).

5

Ah I didn’t know this option, thanks! I’ll try that from now on.

1 Like
6

But if you use pm.trace_to_dataframe(trace), and then pickle the dataframe (say), how do you recover the trace from the dataframe in a form from which you can do a traceplot?

7

I think the best way is to use https://github.com/arviz-devs/arviz
We are currently separating the plotting and statistics module from PyMC3 into a general library, potentially for varies python Bayesian packages like pystan, pyro, edward etc (great work mostly done by @aloctavodia and @colcarroll). The aim is to have something similar to bayesplot or coda in R