I’m trying to participate to the development of PYMC4.
In the meantime I’m getting up to speed with tensorflow probability. (creating a bunch of models in tf prob)
I’m really struggling with convergence issues for Hierarchical models when the priors and initial_chains are mis-specified. E.g. if I initialize my chain even slightly away from where it should be, the convergence will be extremely poor (non-existent for some parameters).
What are the methods used in PYMC3/PYMC4 to deal with this? Do we standardize so that different chains will be equally sensitive to a step? (how to make two gaussian distribution converge when one of them has a large mean and the other a small mean, do we normalize the means so that we’ll have steps of equal importance?).
I have never suffered from convergence issues in PYMC3 before, so I’m assuming there is a lot of work under the hood.
I’m looking at the following notebook, if I set the initial chain to be a bit different from the truth, if won’t converge: