For a task I need to execute Poisson regression. Up until recently, I figured this was no big deal. However, it’s come to my attention that Gradient computations for discrete RVs / PMFs is limited or not supported.
The Gamma PDF does not have the same parameter(s), but it does have continuous shape. Could anyone comment if this is a good/poor choice both from a practical sampling perspective as well as theoretical perspective?
Edit: Perhaps it makes more sense to use exponential or log-normal? My data is events/interval so I figured that Poisson would be most appropriate if gradient computations weren’t an issue but that Gamma was the closest continuous alternative for this sort of data.
You should have no problem with Poisson regression. Gradients are taken of the Poisson log likelihood with respect to the rate parameter \mu which is continuous, not the discrete count observed data y since the data is fixed.
I see, thanks!
I’ve heard from @junpenglao and others that discretes give HMC trouble. I’m quite curious on when this is/isn’t an issue. I frequently find myself asking on a case-by-case basis, having not found a general rule of thumb yet.
If the parameters you’re sampling are discrete you’ll have trouble. Like say you had a model with normally distributed observations, and an unknown mean that was constrained for some reason to be discrete, say 1, 2, or 3. In this scenario NUTS wouldn’t work, but you could still use Metropolis.