ADVI : Collecting Values for Specific Variables

Questions
1

When using ADVI, we can extract the mean and std for free RVs using approx or a tracker. Assume that we collect the latest mean and std vector for free RVs using any of above methods. If we consider the vector that contains means of the all free RVs, then I observed that the position of the value (in the vector) is independent from the order that those variables are specified in the model. In fact, it is impossible to identify the mean of a specific RV given the tracker or approx.

If I want to get the trace of the mean for specif RV from the tracker, how can I collect that ? I appreciate if someone can help me with this.

Thanks

3

See Saving ADVI results and reloading

4

Thanks for the reply, but I don’t see an answer for my question in that post. That post explains how to collect ‘mean’ and ‘rho’. I’m not sure we are in the same page.

Let me explain my problem using an example.

w = Normal('w', mu = 0, sd = 10, shape = 5)
sigma = Normal('sigma', mu  = 0, sd = 10)
mu = w*x 
likelihood = Normal('y' , mu = mu, sd = sigma)

advi = pm.ADVI()
tracker = pm.callbacks.Tracker(
            mean=advi.approx.mean.eval,  # callable that returns mean
            std=advi.approx.std.eval,  # callable that returns std
)
approx = advi.fit(1000, callbacks = [tracker])

Assume that now I want to get the mean for ‘w’ for the last 500th iteration. We can extract the mean of all RVs for 500th iteration using tracker['mean'][500].

But this contains the mean of both w and sigma. Now how can I collect only the mean of w. I can’t use the tracker['mean'][500][0] even though w is specified first in the model because the RVs are reordered during the optimization.

5

I found that we can get an order list of the free RVs according the order they were used during the training. This can be used to get the index of desired free RVs.

model.approx.replacements.keys()

However, this may not be the best way of getting the order of free RVs. I appreciate if someone can point out if there is a better way of doing this.

6

The ordering is saved in bij = advi.approx.groups[0].bij, it has a function to map an array to a dictionary bij.rmap and a mapping from a dictionary to an array bij.map

1 Like
7

Thanks. It works.

I think even model.approx.replacements.keys() does not work for some cases.