Diagnosing divergences when clipping is involved

Questions version agnostic
7

This is great, thank you. So to summarise:

  1. Re-parametrise (this is obviously the most well-documented thing to try and somewhat well understood in the case of Gaussians; less clear once you have non-Gaussians and non-real supports etc)
  2. Look out for distributions with a lot of mass near the boundary (e.g. HalfNormal; see here)
  3. Look out for transformations with regions of flat gradients (e.g. clipping)
  4. Look out for heavy-tailed distributions (e.g. Cauchy, StudentT; this one was sort of new to me since many many tutorials rely on this for “robust regression”)
  5. Write custom samplers (the nuclear button approach, I take it…)
  6. In most cases avoid deep dives into the source of individual divergences (though, in this specific case I found it informative that I could see negative values in the discarded samples whereas there were no negative values in the accepted samples)