# How to save fitted ADVI Result?

**URL:** <https://discourse.pymc.io/t/how-to-save-fitted-advi-result/397>\
**Category:** Questions\
**Created:** [October 6, 2017, 11:48am UTC](https://discourse.pymc.io/t/how-to-save-fitted-advi-result/397 "2017-10-06T11:48:01Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![pstjohn](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/pstjohn/32/268_2.png) [@pstjohn](https://discourse.pymc.io/u/pstjohn)\
**Post date:** [October 6, 2017, 11:48am UTC](https://discourse.pymc.io/t/how-to-save-fitted-advi-result/397/1 "2017-10-06T11:48:01Z")

</div>

_Edit:_, originally a github issue, then posted here, now it’s back to being a github issue [#2615](https://github.com/pymc-devs/pymc3/issues/2615)

If I have a (lengthy) advi optimization, whats the best way to store the result for a future notebook?

```python
with pymc_model:

    approx = pm.fit(
        method='advi', n=50000, random_seed=1, callbacks=cb,
        obj_optimizer=pm.adagrad_window(learning_rate=.5, n_win=100))

    trace = approx.sample(1000)

```

Pickling the trace works,

```python
import pickle

with open('trace.p', 'wb') as f:
    pickle.dump(trace, f)
    
with open('trace.p', 'rb') as f:
    test1 = pickle.load(f)

```

But pickling the approx class fails when I try to load it.

```python
with open('approx.p', 'wb') as f:
    pickle.dump(approx, f)
    
with open('approx.p', 'rb') as f:
    test2 = pickle.load(f)

```

```auto
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-31-54dd7d55c9e6> in <module>()
      3 
      4 with open('approx.p', 'rb') as f:
----> 5 test2 = pickle.load(f)

TypeError: __new__ () missing 1 required positional argument: 'group'

```

## Versions and main components

- PyMC3 Version: 3.1
- Theano Version: 0.9.0
- Python Version: 3.6
- Operating system: Mac OS X
- How did you install PyMC3: from source

---

<div class="post-metadata">

**Author:** ![junpenglao](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/junpenglao/32/8_2.png) [@junpenglao](https://discourse.pymc.io/u/junpenglao)\
**Post date:** [October 6, 2017, 3:00pm UTC](https://discourse.pymc.io/t/how-to-save-fitted-advi-result/397/2 "2017-10-06T15:00:08Z")

</div>

One way to go around it is to initialize the approximation, then do the fit, and then saved the approximation parameters. For example:

```python
with model:
    approx=pm.ADVI()
    approx.fit()
saveparam = [(param.name, param.eval()) for param in approx.approx.params]

```

and load it for later:

```python
with model:
    approx=pm.ADVI()
for i, param in enumerate(saveparam):
    approx.approx.params[i].set_value(param[1])

```

---

<div class="post-metadata">

**Author:** ![junpenglao](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/junpenglao/32/8_2.png) [@junpenglao](https://discourse.pymc.io/u/junpenglao)\
**Post date:** [April 14, 2018, 4:12pm UTC](https://discourse.pymc.io/t/how-to-save-fitted-advi-result/397/3 "2018-04-14T16:12:11Z")

</div>

This is not the safe way to do as showed in [Saving ADVI results and reloading](https://discourse.pymc.io/t/saving-advi-results-and-reloading/1081)  
A better solution is below:

> The safe way to do is to save the parameters as a dictionary, and map it back to a vector whenever the new model is created.

```python
...
# save inference
bij = inference.approx.groups[0].bij
saveparam = {param.name: bij.rmap(param.eval())
	 for param in inference.approx.params}

# new model
model2 = construct_neural_network(
  X, Y, Yerr, neuronsPerHiddenlayer, ninputs, noutputs, ndata)
with model2:
  inference2 = pm.ADVI()

# load inference
bij2 = inference2.approx.groups[0].bij
inference2.approx.params[0].set_value(bij2.map(saveparam['mu']))
inference2.approx.params[1].set_value(bij2.map(saveparam['rho']))

```

---

<div class="post-metadata">

**Author:** ![pstjohn](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/pstjohn/32/268_2.png) [@pstjohn](https://discourse.pymc.io/u/pstjohn)\
**Post date:** [August 6, 2018, 8:35pm UTC](https://discourse.pymc.io/t/how-to-save-fitted-advi-result/397/5 "2018-08-06T20:35:31Z")

</div>

It’s also now possible to pickle ADVI objects and reload them, thanks to a PR from ferrine: [#2619](https://github.com/pymc-devs/pymc3/pull/2619)