The need for reparametrization is quite a common problem I suppose.
I read the nice paper by Betuncourt, Girolami(2013) which deals with funnels for hierarchical models and I think I understand now that the general problem for Euclidian HMC (and other samplers excluding Riemannian HMC) are strong local (on small scales) correlations which significantly differ from global correlations on different, typically larger scales. These features of parameter space are typically called “funnels”.
I understand that a common reparametrization choice for hierarchical models it to go from centered to non-centered parametrizations. But what about a more general case which also features funnels?
In section 12.1 of the Baysian Data Analysis book Gelman seems to suggest a more general approach of parameter expansion through the addition of auxiliary parameters. The selection of additional parameters are guided by the conditional distributions of parameters between each other, if some dependency could induce strong shrinkage, it is offset (or multiplied, i.e. offset in logarithmic space) by some independent auxiliary parameter. This is mostly useful for the Gibbs sampler though.
Could you recommend some resources or common practice how to reparametrize towards reducing funnels in general?