PyMC3 -> PyMC v4 migration

The new v4 of PyMC3 will be renamed as PyMC v4. everything is still getting figured out. Everyone at @PyMC_devs wants to hear about worries, concerns, doubts or suggestions about this that you might have. We’d love to take care of things beforehand so the community doesn’t struggle with the change. Very soon a migration guide document will be released to help you out with all of this. You can find some updates in this PR and Github Project for the meantime:

3 Likes

Can you change the title to say “PyMC”?

tried but title has a lower limit of 15 characters

I meant just to rename “Pymc3 → Pymc v4 migration” to “PyMC3 → PyMC v4 migration”

1 Like

Sounds like a good move to me. I’ll have to update some of my material, but I don’t have any concerns.

3 Likes

Massive work - thanks to all involved.

I see my proposal for PyMC++ and then PyMC# wasn’t adopted disappointingly :rofl:

On a more serious note: given the quite different APIs for v3 versus v4, where do you want new PRs to come? I presume targetting v3 would only lead to more work to port to v4, so should contributors hold off for a bit?

1 Like

We have halted new features to V3. Everything should now go towards V4, unless it’s some sort of major bug fix

1 Like

I am very excited about this development. Improved performance is always nice, especially given how massive some improvements seem to be. Another convenience that will hopefully make my live much easier is that then C-compilation will be dropped. The compilation directory had to be managed manually when multiple projects ran on the same file system and they could grow to terabytes in size. All this will probably go away and make working with PyMC even easier.

My main concern is that less commonly used features will go away in the new version. Specifically, using the .dist function of distributions without generating a named RV seems to work differently and I do not know how yet. I know that using the distributions in pm.Potential is sometimes considered “hacky”. But if you know the analytic form of the posterior, it can be more straight forward to use it in pm.Potential and in some use cases I have not found an alternative way. Another maybe less loved feature is the find_MAP since its results can be misleading and some of the advantages of Bayesian analysis are wasted. However, if one makes sure the posterior is concave and its mode aligns with the mean, then there is no quicker way to find it.

PyMC v4 seems to be an amazing advancement for the software in any case and I am very grateful for the awesome community and the extremely useful software! All the best for the lunch :rocket:

3 Likes

hi @Dominik ,
while Aesara is giving you more options for compilation backends (numba, jax) and making it easier to add more, the C backend will likely remain supported for the foreseable future. Mainly because the other backends don’t have the full support/compatibility with all scenarios or OSes.
But Aesara also got considerable refactoring of how it’s managing the compilation directory, so hopfully it’ll be less of a headache. aesara.compiledir also has cleanup() and compiledir_purge() that you might be interested in.

It’s correct that the .dist() API changed, but these changes should actually make it more compatible with pm.Potential, pm.logp() etc. Long story short. The Distribution() API implements niceties related to dims/coords management in the model, but inside creates the RVs with Distribution.dist().
Additionally, we’re (going to) using Aeppl for the construction of log-probability graphs, which reduces our dependency on information provided by Distribution()/Model.