I think this point has been made several times along the thread, but you really need to look at the prior predictive to know if your model is behaving as expected. If it does not generate the data you expect to see, then there is no way for your posterior to be any different. You seem to be unhappy with the high probability of zero in your PPC. However, as this is completely expected with an exponential. You can certainly normalize to per 100k and then use a different distribution, including gamma. However, again, depending on your priors for alpha/beta, you may still have a model with high probability near zero. It seems that in the model link you posted above, you are defining alpha=1. This is likely inconsistent with that you expect from your data. Are you expecting something that looks more like a normal distribution centered around some positive value? Take a look at the visualization and see what you think:
https://www.pymc.io/projects/docs/en/stable/api/distributions/generated/pymc.Gamma.html
Do you want a fixed value of alpha or would you consider placing priors on both alpha/beta? You can also parameterize the gamma using mu/sigma if that makes more sense for your application.