I’m currently working on a personal project using ADVI and there are a couple of quality of life improvement I am looking for.
I’m considering creating these myself as a PR for PyMC, but before I start my deep dive into the inner workings of PyMC I would like to know if any such PR would get reviewed and has at least a chance of getting accepted.
Extract mean and variance of the fitted parameters directly without resorting to sampling the posterior. Some code snippets for earlier versions use bij for this, but bij no longer exist in the current version.
Use the posterior of a fit to one data set as the prior for the fit to another data set
Set a starting value for the sigmas (not just the means)
I know pymc/CONTRIBUTING.md at main · pymc-devs/pymc · GitHub , Is there anything else I should be aware of?
1 Like
schmaus:
Extract mean and variance of the fitted parameters directly without resorting to sampling the posterior. Some code snippets for earlier versions use bij for this, but bij no longer exist in the current version.
Sounds useful, I’ve seen people ask for this repeatedly
schmaus:
Use the posterior of a fit to one data set as the prior for the fit to another data set
I think this was implemented already in pymc-experimental: https://github.com/pymc-devs/pymc-experimental/pull/56
schmaus:
Set a starting value for the sigmas (not just the means)
Can’t say as I have not used VI. CC @fonnesbeck @ferrine
2 Likes
I had just run into the same issue with extracting means and variances yesterday, so this is timely! Yes, I think the ADVI user API could use some upgrades, for sure. Please do check out what is already in pymc-experimental, as Ricardo noted.
I think sigmas can be set via the rho shared parameter, but I agree its not currently designed to be set by the user.
In any case, I guarantee your PR would be reviewed, and I don’t see any reason it would not be merged in some form.
3 Likes
This PR was about converting traces to priors, but not approximations to priors. An alternative PR is welcome in pymc-experimental
3 Likes
I started working on extracting mean and std from the posterior, since this is the simplest task.
pymc-devs:main ← markusschmaus:advi_mean_data
opened 12:55PM - 31 Aug 22 UTC
**What is this PR about?**
For VI the mean and std of approximations is current… ly only available as an unstructured flat Aesara Variable. This leads to frequent questions on how to extract these properties from the posterior. This PR creates two new properties which evaluate the Aesara Variables and transforms them into a structured xarray Dataset using the available `coords`.
**Checklist**
+ [ ] Explain important implementation details 👆
+ [ ] Make sure that [the pre-commit linting/style checks pass](https://docs.pymc.io/en/latest/contributing/python_style.html).
+ [ ] Link relevant issues (preferably in [nice commit messages](https://tbaggery.com/2008/04/19/a-note-about-git-commit-messages.html))
+ [ ] Are the changes covered by tests and docstrings?
+ [ ] Fill out the short summary sections 👇
## Major / Breaking Changes
- ...
## Bugfixes / New features
- ...
## Docs / Maintenance
- ...
2 Likes
schmaus
September 3, 2022, 11:16am
6
Here is the next for allowing to provide a starting value for sigma:
pymc-devs:main ← markusschmaus:advi_start_sigma
opened 06:09PM - 02 Sep 22 UTC
Introduces the `start_sigma` argument which allows to set a starting value for t… he sigmas of mean field approximation in `ADVI`. I am using the `ordering` property to create the mapping between the variables and the flat 1d array required by `ADVI`.
See also:
https://discourse.pymc.io/t/quality-of-life-improvements-to-advi/10254
**Checklist**
+ [x] Explain important implementation details 👆
+ [x] Make sure that [the pre-commit linting/style checks pass](https://docs.pymc.io/en/latest/contributing/python_style.html).
+ [x] Link relevant issues (preferably in [nice commit messages](https://tbaggery.com/2008/04/19/a-note-about-git-commit-messages.html))
+ [x] Are the changes covered by tests and docstrings?
+ [x] Fill out the short summary sections 👇
## Major / Breaking Changes
- None
## Bugfixes / New features
- Allow `start_sigma` being given to `fit`.
- For `ASVGD` forward the key word arguments `start` and `random_seed` to the constructor of the default approximation `FullRank` instead of sending them to `super().__init__` which can't process them
- Enable `_iterate_with_loss` to run with `n=0` by giving an appropriate logger message. This allows to check initialization values, which is useful for testing, but could also be useful for debugging a user model
## Docs / Maintenance
- The documentation for the `start` argument was outdated, it claimed type `Point`, but it gets passed through to a function that requires `StartDict`. Since users are probably unfamiliar with this custom type (`Dict[Union[Variable, str], Union[np.ndarray, Variable, str]]`), I decided to also mention the most relevant and most commonly used subtype `dict[str, np.ndarray]`.
1 Like
schmaus
September 6, 2022, 1:34pm
7
I know you are all pretty busy, but just so I know what to expect. What is the time frame for getting a review on these PRs?