# Unable to load trace after VI

**URL:** <https://discourse.pymc.io/t/unable-to-load-trace-after-vi/3973>\
**Category:** Questions\
**Tags:** variational\_inferenc\
**Created:** [October 16, 2019, 1:09am UTC](https://discourse.pymc.io/t/unable-to-load-trace-after-vi/3973 "2019-10-16T01:09:58Z")\
**Posts on this page:** 15\
**Page:** 1

<div class="post-metadata">

**Author:** ![SosUts](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/sosuts/32/1906_2.png) [@SosUts](https://discourse.pymc.io/u/SosUts)\
**Post date:** [October 16, 2019, 1:09am UTC](https://discourse.pymc.io/t/unable-to-load-trace-after-vi/3973/1 "2019-10-16T01:09:58Z")

</div>

The following simple code does not work.  
How can I save the results after VI?  
`pm.save_trace()` and `pm.load_trace()` works perfectly at MCMC traces.  
Thank you in advance.

```auto
with pm.model() as model:
    #simple model
    
with model:
    advi = pm.ADVI()
approx = advi.fit()
mytrace = approx.sample(100000)
# mytrace is <MultiTrace: 1 chains, 100000 iterations, 14 variables>

pm.save_trace(mytrace, "Some_kind_of_directory")
mytrace2 = pm.load_trace("Some_kind_of_directory", model=model)

```

returns  
`TypeError: 'NoneType' object is not iterable`

---

<div class="post-metadata">

**Author:** ![junpenglao](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/junpenglao/32/8_2.png) [@junpenglao](https://discourse.pymc.io/u/junpenglao)\
**Post date:** [October 16, 2019, 6:13am UTC](https://discourse.pymc.io/t/unable-to-load-trace-after-vi/3973/2 "2019-10-16T06:13:29Z")

</div>

@ferrine do you know if VI supports this?

---

<div class="post-metadata">

**Author:** ![SosUts](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/sosuts/32/1906_2.png) [@SosUts](https://discourse.pymc.io/u/SosUts)\
**Post date:** [October 16, 2019, 1:49pm UTC](https://discourse.pymc.io/t/unable-to-load-trace-after-vi/3973/3 "2019-10-16T13:49:12Z")

</div>

I totally thought MultiTrace object of VI is almost identical to that of MCMC…  
I was wondering why everyone use pickle.

---

<div class="post-metadata">

**Author:** ![junpenglao](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/junpenglao/32/8_2.png) [@junpenglao](https://discourse.pymc.io/u/junpenglao)\
**Post date:** [October 16, 2019, 1:55pm UTC](https://discourse.pymc.io/t/unable-to-load-trace-after-vi/3973/4 "2019-10-16T13:55:44Z")

</div>

For most part, it should behave like a MultiTrace…

---

<div class="post-metadata">

**Author:** ![SosUts](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/sosuts/32/1906_2.png) [@SosUts](https://discourse.pymc.io/u/SosUts)\
**Post date:** [October 16, 2019, 2:24pm UTC](https://discourse.pymc.io/t/unable-to-load-trace-after-vi/3973/5 "2019-10-16T14:24:52Z")

</div>

There should be some problems in my local environment.  
I’ll try the code in other pc or google colab.

Thank you as always!

---

<div class="post-metadata">

**Author:** ![SosUts](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/sosuts/32/1906_2.png) [@SosUts](https://discourse.pymc.io/u/SosUts)\
**Post date:** [October 31, 2019, 7:24am UTC](https://discourse.pymc.io/t/unable-to-load-trace-after-vi/3973/6 "2019-10-31T07:24:23Z")

</div>

Still, I can`t `pm.load_trace()`…

```auto
Windows10, Corei7-8700 (3.2GHz) 
RAM : 16 GB

Python : 3.7.4
pymc3 : 3.6
arviz : 0.5.1

```

Installed Python and pymc3 by Anaconda, arviz by pip.

I have already tried updating to pymc3 to 3.7, re-installing python and every packages.  
Updating to 3.7 caused another problem : slowing `pm.traceplot()` speed significantly.  
It took more than 5 min to plot the result of a very simple model.

Have anyone experienced similar problems?

---

<div class="post-metadata">

**Author:** ![junpenglao](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/junpenglao/32/8_2.png) [@junpenglao](https://discourse.pymc.io/u/junpenglao)\
**Post date:** [October 31, 2019, 8:01am UTC](https://discourse.pymc.io/t/unable-to-load-trace-after-vi/3973/7 "2019-10-31T08:01:42Z")

</div>

What error are you seeing?

As for slowness, likely because of an older version of Arviz - did you also upgraded that?

---

<div class="post-metadata">

**Author:** ![SosUts](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/sosuts/32/1906_2.png) [@SosUts](https://discourse.pymc.io/u/SosUts)\
**Post date:** [October 31, 2019, 8:47am UTC](https://discourse.pymc.io/t/unable-to-load-trace-after-vi/3973/8 "2019-10-31T08:47:50Z")

</div>

As for `pm.save_trace`, same error as the [post above](https://discourse.pymc.io/t/unable-to-load-trace-after-vi/3973).  
`'NoneType' object is not iterable`

There was a similar post in September.  
[Unable to load trace](https://discourse.pymc.io/t/unable-to-load-trace/3908)

* * *

arviz 0.5.1 seems to be the latest version.  
The following codes are the things I tried just now.  
Both resulted in slow very `pm.traceplot`.

1: Reinstall arviz via pip.

```python
pip uninsall arviz
pip install git+https://github.com/arviz-devs/arviz

```

2: Reinstall arviz via conda

```python
pip uninstall arviz 
conda uninstall arviz
conda install -c conda-forge arviz

```

---

<div class="post-metadata">

**Author:** ![junpenglao](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/junpenglao/32/8_2.png) [@junpenglao](https://discourse.pymc.io/u/junpenglao)\
**Post date:** [October 31, 2019, 10:09am UTC](https://discourse.pymc.io/t/unable-to-load-trace-after-vi/3973/9 "2019-10-31T10:09:01Z")

</div>

Thanks for reporting back - could you share your model as well? We will try to have take a look at the slowness in Arviz.  
Meanwhile, @ferrine I think the load\_trace breakage is a bug, have you seen it before?

---

<div class="post-metadata">

**Author:** ![junpenglao](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/junpenglao/32/8_2.png) [@junpenglao](https://discourse.pymc.io/u/junpenglao)\
**Post date:** [October 31, 2019, 10:13am UTC](https://discourse.pymc.io/t/unable-to-load-trace-after-vi/3973/10 "2019-10-31T10:13:53Z")

</div>

For now, try [Saving ADVI results and reloading](https://discourse.pymc.io/t/saving-advi-results-and-reloading/1081) to save the approximation instead of the trace.

---

<div class="post-metadata">

**Author:** ![SosUts](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/sosuts/32/1906_2.png) [@SosUts](https://discourse.pymc.io/u/SosUts)\
**Post date:** [October 31, 2019, 10:59am UTC](https://discourse.pymc.io/t/unable-to-load-trace-after-vi/3973/11 "2019-10-31T10:59:16Z")

</div>

This is my model.  
Data size is about 30k, and it is circular data.

```python
with pm.Model() as model:
    kappa_1 = pm.Gamma('kappa_1', 1., 1., )
    mu_1 = pm.VonMises('mu_1', mu=0, kappa=0.5)
    component1 = pm.VonMises.dist(mu=mu_1, kappa=kappa_1)
    kappa_2 = pm.Gamma('kappa_2', 1., 1., )
    mu_2 = pm.VonMises('mu_2', mu=np.pi, kappa=0.5)
    component2 = pm.VonMises.dist(mu=mu_2, kappa=kappa_2)

    w = pm.Dirichlet('w', np.ones_like([1., 1.,]))
    vm = pm.Mixture('vm', w=w, comp_dists=[component1, component2,],
                    observed=data.values, )
with model:
    fullrank = pm.FullRankADVI()
tracker = pm.callbacks.Tracker(
    mean = fullrank.approx.mean.eval, # callable that returns mean
    std = fullrank.approx.std.eval # callable that returns std
)
approx = fullrank.fit(100000, callbacks=[tracker])

pm.save_trace(trace=approx.sample(100), directory="test", overwrite=True)
trace = pm.load_trace("test", model=model)
# returns 'NoneType' object is not iterable

```

* * *

## pymc3:3.7

I found out that this is some kind of memory problem.  
Elapsed time is very short but it took few minutes for the plot to appear in my jupyter notebook.  
Increasing sampling number to `approx.sample(10000)` returned `MemoryError`.  
This didn’t happen in pymc3 3.6.0.

```python
import time
pm.traceplot(approx.sample(1000))
start = time.time()
elapsed_time = time.time() - start
print ("elapsed_time:{0}".format(elapsed_time) + "[sec]")
# elapsed_time:0.0029921531677246094[sec]

```

## pymc3:3.6

**Plots appeared in few seconds.**

```python
import time
pm.traceplot(approx.sample(1000))
start = time.time()
elapsed_time = time.time() - start
print ("elapsed_time:{0}".format(elapsed_time) + "[sec]")
# elapsed_time:0.0[sec]
```

---

<div class="post-metadata">

**Author:** ![SosUts](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/sosuts/32/1906_2.png) [@SosUts](https://discourse.pymc.io/u/SosUts)\
**Post date:** [October 31, 2019, 11:03am UTC](https://discourse.pymc.io/t/unable-to-load-trace-after-vi/3973/12 "2019-10-31T11:03:54Z")

</div>

> For now, try [Saving ADVI results and reloading](https://discourse.pymc.io/t/saving-advi-results-and-reloading/1081) to save the approximation instead of the trace.

OK! Thank you!  
I really appreciate your reply.  
I don’t know how to apologize wasting your time if this is a problem of my local environment…

---

<div class="post-metadata">

**Author:** ![junpenglao](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/junpenglao/32/8_2.png) [@junpenglao](https://discourse.pymc.io/u/junpenglao)\
**Post date:** [October 31, 2019, 1:49pm UTC](https://discourse.pymc.io/t/unable-to-load-trace-after-vi/3973/13 "2019-10-31T13:49:15Z")

</div>

Happy to help and we appreciate your feedback! @RavinKumar could you check if you could reproduce the slowness issue?  
In general, since we are using the same plotting backend, I would expect there is no slowdown, but I could be wrong

---

<div class="post-metadata">

**Author:** ![SosUts](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/sosuts/32/1906_2.png) [@SosUts](https://discourse.pymc.io/u/SosUts)\
**Post date:** [November 25, 2019, 4:53pm UTC](https://discourse.pymc.io/t/unable-to-load-trace-after-vi/3973/14 "2019-11-25T16:53:15Z")

</div>

Was the slowness reproducible?

---

<div class="post-metadata">

**Author:** ![Safety\_Mary](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/safety_mary/32/4598_2.png) [@Safety\_Mary](https://discourse.pymc.io/u/Safety_Mary)\
**Post date:** [November 24, 2021, 4:17am UTC](https://discourse.pymc.io/t/unable-to-load-trace-after-vi/3973/15 "2021-11-24T04:17:24Z")

</div>

I also encounter the same problem. I try running the code on  
[https://docs.pymc.io/en/stable/pymc-examples/examples/variational\_inference/bayesian\_neural\_network\_advi.html](https://docs.pymc.io/en/stable/pymc-examples/examples/variational_inference/bayesian_neural_network_advi.html)

then do  
pm.save\_trace(trace, ‘test\_trace’)  
pm.load\_trace(directory=‘test\_trace’, model=neural\_network)

results in the following error while pm.load\_trace  
TypeError: ‘NoneType’ object is not iterable

I tried on Colab, my local machine and AWS, all resulting in the same error.