Quality of life improvements to ADVI

I’m currently working on a personal project using ADVI and there are a couple of quality of life improvement I am looking for.

I’m considering creating these myself as a PR for PyMC, but before I start my deep dive into the inner workings of PyMC I would like to know if any such PR would get reviewed and has at least a chance of getting accepted.

  • Extract mean and variance of the fitted parameters directly without resorting to sampling the posterior. Some code snippets for earlier versions use bij for this, but bij no longer exist in the current version.
  • Use the posterior of a fit to one data set as the prior for the fit to another data set
  • Set a starting value for the sigmas (not just the means)

I know pymc/CONTRIBUTING.md at main · pymc-devs/pymc · GitHub, Is there anything else I should be aware of?

1 Like

Sounds useful, I’ve seen people ask for this repeatedly

I think this was implemented already in pymc-experimental: Initialize a prior from a fitted posterior by ferrine · Pull Request #56 · pymc-devs/pymc-experimental · GitHub

Can’t say as I have not used VI. CC @fonnesbeck @ferrine

2 Likes

I had just run into the same issue with extracting means and variances yesterday, so this is timely! Yes, I think the ADVI user API could use some upgrades, for sure. Please do check out what is already in pymc-experimental, as Ricardo noted.

I think sigmas can be set via the rho shared parameter, but I agree its not currently designed to be set by the user.

In any case, I guarantee your PR would be reviewed, and I don’t see any reason it would not be merged in some form.

3 Likes

This PR was about converting traces to priors, but not approximations to priors. An alternative PR is welcome in pymc-experimental

3 Likes

I started working on extracting mean and std from the posterior, since this is the simplest task.

2 Likes

Here is the next for allowing to provide a starting value for sigma:

1 Like

I know you are all pretty busy, but just so I know what to expect. What is the time frame for getting a review on these PRs?

Replied on the PR.