I’m trying to fit ADVI approximation for my model, and at first it works:

``````    advi = pm.ADVI()
trace = approx.sample()
``````

gives me trace corresponding to values after 10000 ADVI iterations. Then I call `advi.refine(100000)`, but after it `approx` and `advi.approx` still correspond to the state after first 10000 iterations. So, how to `sample` after `refine`ing?

You are right, the `refine` doesnt seems to doing what it suppose to - for what is worth, you can call `advi.fit(100000)` instead of `refine`, which it will continue training and resulting a more refine approximation.

So, am I right that

``````    approx = advi.fit(10000)
trace = approx.sample()
``````

and then again calling

``````    approx = advi.fit(10000)
trace = approx.sample()
``````

would lead to `trace` corresponding to the state after 20000 iterations? And `approx.hist` will have 20000 as well, correct?

UPD: yeah, confirm that this is the case. Thank you, I thought that `.fit` starts from scratch. So, does `refit()` do something useful?

1 Like

It seems not very useful - it was not documented and there is no test case.
We should probably remove it.

The place where I found `refine` was https://docs.pymc.io/notebooks/variational_api_quickstart.html btw.

Yeah I think there is a bug some where that the state is not updated in `advi.approx`. As when you call `.refine`, `advi.hist` will change but not `advi.approx.hist`.