How to initialize ADVI

Questions v5
1

How can I set the starting point of ADVI for M1 using the meanfield approximation from M2? I saw that pm.ADVI has two start arguments but couldn’t understand how to populate them.

with M2:
    
  inference = pm.ADVI(random_seed=seed)

  approx = pm.fit(n= 10000, random_seed=seed, method=inference, obj_optimizer=adam(learning_rate=0.05,decay_iter=4000))
  idata = approx.sample(2000,random_seed=seed) 

with M1:
  inference = pm.ADVI(random_seed=seed, start=??)


  approx = pm.fit(n= 10000, random_seed=seed, method=inference, obj_optimizer=adam(learning_rate=0.05,decay_iter=4000))
  idata_servicer = approx.sample(2000,random_seed=seed)

I looked at How to choose initial values for MeanField sigma, and transform rho->sigma?, but the solution there did not work. Any suggestions @ricardoV94 @jessegrabowski?

1 Like
2

I think @fonnesbeck uses VI on Mac. Maybe he has some suggestions?

3

The start argument expects a dict of starting values for the random variables in the model. So if you had a linear regression model it would be something like: start={"intercept": 0, "slope": 1, "noise": 3}. So if you are populating it with the results from another model, you could put the posterior means in there, for example.

2 Likes