Adaptive MC w/ normalizing flows

Cool! It seems related (in spirit) to the NeuTra HMC paper from a few years ago ([1903.03704] NeuTra-lizing Bad Geometry in Hamiltonian Monte Carlo Using Neural Transport). My 2 cents: attacking difficult posterior distributions this way is really helpful when you have moderate dimensionality (100 < d < 10000) and high correlations. My (limited) knowledge on this is that the compute cost incurred by the NF is at least linear in d so the payoff might not always be worth it.

Unfortunately, this is also kind of a mismatch to the problem space many advanced modelers face because you either have something with d >> 10000 like regression or Bayesian machine learning with many groups / covariates, or you have small d with extremely difficult posterior geometries from highly nonlinear models like ODE/PDEs from physics/ecology/etc in which case you can use Riemannian HMC.

1 Like