Using ADVI refine

Questions
1

Iā€™m trying to fit ADVI approximation for my model, and at first it works:

    advi = pm.ADVI()
    approx = advi.fit(10000)
    trace = approx.sample()

gives me trace corresponding to values after 10000 ADVI iterations. Then I call advi.refine(100000), but after it approx and advi.approx still correspond to the state after first 10000 iterations. So, how to sample after refineing? :slight_smile:

2

You are right, the refine doesnt seems to doing what it suppose to - for what is worth, you can call advi.fit(100000) instead of refine, which it will continue training and resulting a more refine approximation.

3

So, am I right that

    approx = advi.fit(10000)
    trace = approx.sample()

and then again calling

    approx = advi.fit(10000)
    trace = approx.sample()

would lead to trace corresponding to the state after 20000 iterations? And approx.hist will have 20000 as well, correct?

UPD: yeah, confirm that this is the case. Thank you, I thought that .fit starts from scratch. So, does refit() do something useful?

4

It seems not very useful - it was not documented and there is no test case.
We should probably remove it.

5

The place where I found refine was https://docs.pymc.io/notebooks/variational_api_quickstart.html btw.

6

Yeah I think there is a bug some where that the state is not updated in advi.approx. As when you call .refine, advi.hist will change but not advi.approx.hist.