Evaluating Variational Inference: Need Access to fitted (AD)VI approximation density

Questions
2

After you fit a model using ADVI, you can sample from the approximation:

with NonCentered_eight:
    ADVI1 = pm.fit(20000, method='advi', obj_n_mc=100,
                   obj_optimizer=pm.adagrad(learning_rate=0.1))
    noncentered_ADVI = ADVI1.sample(10000)

Resulting noncentered_ADVI which you can just treat it as a trace object from PyMC3 that contains MCMC samples. Then you evaluate the model logp using each slice of the samples (we call it point) as input:

logp_func = NonCentered_eight.logp
for point in noncentered_ADVI:
    p_theta_y.append(logp_func(point))

If you are stucked, you can have a look at my partial port of the said paper here: https://github.com/junpenglao/Planet_Sakaar_Data_Science/blob/master/WIP/[WIP]%20Comparing%20VI%20approximation.ipynb

1 Like