New paper turned up on PNAS about normalizing flows in Monte Carlo,

- https://www.pnas.org/doi/epdf/10.1073/pnas.2109420119
- GitHub - marylou-gabrie/flonaco (pytorch impl)

May be of interest?

New paper turned up on PNAS about normalizing flows in Monte Carlo,

- https://www.pnas.org/doi/epdf/10.1073/pnas.2109420119
- GitHub - marylou-gabrie/flonaco (pytorch impl)

May be of interest?

1 Like

Cool! It seems related (in spirit) to the NeuTra HMC paper from a few years ago ([1903.03704] NeuTra-lizing Bad Geometry in Hamiltonian Monte Carlo Using Neural Transport). My 2 cents: attacking difficult posterior distributions this way is really helpful when you have moderate dimensionality (100 < d < 10000) and high correlations. My (limited) knowledge on this is that the compute cost incurred by the NF is at least linear in d so the payoff might not always be worth it.

Unfortunately, this is also kind of a mismatch to the problem space many advanced modelers face because you either have something with d >> 10000 like regression or Bayesian machine learning with many groups / covariates, or you have small d with extremely difficult posterior geometries from highly nonlinear models like ODE/PDEs from physics/ecology/etc in which case you can use Riemannian HMC.

1 Like

Thanks for the good remarks. I agree that it’s not a giant hammer to squash all problems. But I suspect in complex models it could help address certain difficult or inexistant reparametrizations.

Slightly tangential question, but how often you you think sampling difficulty arises from multi-modality, as opposed to difficult or degenerate posterior geometry? In my own work (economics) I see the latter much more often than the former, although I admittedly don’t work much with mixture models.

Multimodality is definitely less of an issue than it used to be, since the implementation of features like the `ordered`

transform and mixture classes have made label switching much less common. Variational inference has also been pretty helpful too, since it typically just picks a single mode for the approximate posterior though that’s more or less just shoving the issue under the rug so it doesn’t show up in the convergence diagnostics.

1 Like