How to save mcmc resulte

Questions
1

Hi,
I want to save the sample to the local file,

with model:
    start=pm.find_MAP(model=model, fmin=optimize.fmin_powell)
    trace=pm.sample(50,start=start,njobs=1,trace=pm.backends.text.Text('try'))

but there was a mistake

ValueError: Chains are not unique.

I hope to get your advice, thank you.

2

I think the issue pertains to the fact that you are starting all your sampling runs from the same starting point - checking lack of convergence on the basis of the rhat statistic later doesn’t make sense if all the chains started from the same starting point. In addition, you are sampling just 50 samples - likely those 50 samples end up being very similar in all your chains. Usually, it is not advisable to start sampling at the MAP (especially in high dimensions) - it seems like the recommendation is to random starts and run several chains and check lack of convergence with the rhat (which pymc3 automatically does).

3

Unfortunately this is caused by a bug in the text backend:

@colcarroll recently implemented a new load/save mechanism, we dont have an example yet but you can try follow the test case:

4

Thank you two for helping me to puzzle out, and If I want to save the sampling results to the Excel file, how do I operate it?Is there such an example here？

5

You can use pm.trace_to_dataframe to save the trace into a dataframe, and then save it as a .csv file using pandas.

1 Like
6

I want to save multiple sampling results in different workbooks of the same Excel file：

write1=pd.ExcelWriter('w.xlsx')
df=[]
for i in range(M):
    with pm.Model() as model:
        packed_l=pm.LKJCholeskyCov('packed_l',n=2,eta=2,sd_dist=pm.HalfCauchy.dist(2.5))
        L=pm.expand_packed_triangular(2,packed_l)
        Sigma=pm.Deterministic('Sigma',tt.dot(L,L.T))
        mu=pm.Normal('mu',0.,10,shape=2,testval=dataSet[i].mean(axis=0))
        obs=pm.MvNormal('obs',mu=mu,chol=L,observed=dataSet[i])
        trace = pm.sample(random_seed=seed,model=model,njobs=1)
    
    df.append(pm.trace_to_dataframe(trace).to_excel(write1,sheet_name='sheet'+str(i)))

But after the sampling, the w.xlsx is empty, and the result is not saved.

7

First, Do you call write1.close() ?

If you call that, then this should work, unless your trace it empty. Did you debug and check the trace?

I tried reproducing the issue but, it is difficult to generate data for your model. The logic you used to save the trace works. I tried the same logic with different model and it worked. May be the problem is with the model.

2 Likes
8

Thank you very much for your reminding， I called write1.close() before. It’s really a problem with the model， after I modify it, I can store the sampling results.