I am working on a ~weekly series on modern implementations of gradient based samplers. Three parts are out so far, and I am happy to discuss them here, or in the issues of the github repo that accompanies the articles (minimc).
This series is strongly influenced by PyMC3’s implementation, and I am using it as a testbed of ideas for PyMC4 and improvements to PyMC3.
I think these posts are great. From a basic perspective (like mine), they are quite informative for two reasons: the graphs are well-made and used effectively (and we should all remember how much more efficient a single good graph can be at communicating complex data, versus a million words), and your explanations are concise.
I would like to comment further and more deeply on these posts, but alas I am still a MCMC beginner; but, even as a beginner, these posts were useful in conceptualizing and elaborating the steps of advanced MC schemes.
New post here on higher order integrators: what they are, and why don’t we use them?
As a historical note, they exist in the PyMC3 codebase, implemented in theano, but we switched the integrators to numpy soon after, and Bob Carpenter of Stan said the higher order integrators never really helped in practice. I think no one has the heart to get rid of them or document how to use them and when.