Did anybody replicate the ADVI examples in the "Automatic Variational Inference in Stan" paper?

Questions
1

Hello,

I was wondering if somebody already replicated the examples from the “Automatic Variational Inference in Stan” paper (https://arxiv.org/abs/1506.03431) in PyMC3?

I would appreciate a link to those examples.

Thanks a lot,
Christian

2

I dont think so. But it should be quite straight forward to implement.
Contribution welcome as doc!

3

Ok, I can do that. But I have already some problems with some basic constructs, e.g. “Figure 8: Stan code for Gamma Poisson non-negative matrix factorization model.” uses the Stan positive_ordered vector constraint, where all entries are constrained to be positive and ordered. How could I achieve similar effects in PyMC3?

I’ll start to work on the examples and will publish a link to them once they’re in good shape.

4

Since the theta in Figure 8 is generated from a Gamma distribution, it is already positive. In PyMC3 you just need to specify the right shape, and then tt.sort it along the right axis.

You can also try to use an order transformation (by @aseyboldt), it’s not in our code base yet so if it doesnt work ask @aseyboldt :wink:

1 Like
5

Hi all,

I might be little late to the party, but I’m also working on the replication of the examples @cs224 mentioned.
I haven’t figured out a way to sort the matrix directly, thus I create the vector sequentially and stack them later. This leads however to very long fitting times.

Does anyone have an idea to improve upon this?

Thanks in advance!
Hendrik

with pm.Model() as pmf:
    U_model = pm.Gamma('U', alpha=alpha_prior, beta=0.89, shape=(df.shape[0], n_components))

    ordered_vectors = []
    for i in range(df.shape[1]):
        vector = pm.Gamma(f'vector_{i}', alpha=1, beta=1, shape=n_components,
                          transform=pm.distributions.transforms.ordered)
        ordered_vectors.append(vector)

    V_model = tt.stack(ordered_vectors, axis=0)

    R = pm.Poisson('R', mu=pm.math.dot(U_model, V_model.T), observed=df)

with pmf:
    inference = pm.ADVI()
    callback = pm.callbacks.CheckParametersConvergence(tolerance=1e-2)
    approx = pm.fit(n=10_000,
                    method=inference,
                    callbacks=[callback, print_callback])
    trace = approx.sample(10_000 // 2)