Usage of Grouped ADVI

Questions
1

Hi everybody,

Hopefully, this is a quick question but I am a little confused about the API for setting up a grouped approximation with minibatch ADVI.

I’ve specified 2 groups using the Group within my model context but I don’t understand how to pass the Approximation to fit the model? Here’s a skeleton version of what my model is doing:

n =1000 # data size
Y=theano.shared(Y_data, shared = True)
X=theano.shared(X_data, shared = True)
with Model() as model:
    minibatch_y = pm.Minibatch(data =Y.get_value() , batch_size = 300)
    minibatch_x = pm.Minibatch(data = X.get_value() , batch_size = 300)
    # priors
    beta = pm.Normal('beta', np.zeros(2), np.ones(2)*100, shape = (3,2))       
    theta = pm.Normal("theta",0,0.001)
    def logp(x,y):
          return f(theta, beta,x,y) # some likelihood of theta and beta
    y_obs = pm.DensityDist("y_obs", logp, observed={'x':minibatch_x, 'y':minibatch_y}, total_size =n)
    ### Now define groups
    group_fr = pm.Group([beta], vfam="fr") # make approx for betas full rank
    group_mf = pm.Group([None], vfam="mf") # the rest mean field
    approx = pm.Approximation([group_fr, group_mf])

At this point without the groups, I would run something like

with model:
      advi_fit = pm.fit(n=10000, method ="fullrank_advi")

Now that the groups are set up, how/where do I pass the “approx” containing the groups to fit the model?

Thanks!

David

2

You are right, this part is not clear in the doc. You can do:

with model:
    advi_fit = pm.KLqp(approx).fit(n=10000)

e.g., see in the test:

1 Like
3

Awesome, thanks! I knew it had to be something simple like that. I appreciate it