If you have probability mass piling up at zero, it’s saying something about your data’s compatibility with your model (prior and likelihood). You can sweep it under the rug by adding 1e-9 or clipping (which aren’t quite the same density, but very close, and the addition is much more numerically stable). Or you could try to get to the bottom of why the data’s consistent with a value of zero for that parameter. Usually it’s because you have a partial pooling model and the data’s compatible with complete pooling. Depending on how much probability mass is piled up near zero, you might want to just switch to a complete pooling model in those situations.
The choice of a half normal or half Cauchy is intentionally allowing positive density at a value of zero in order to allow for complete pooling. But it can be a computational nightmare due to underflow in floating point caclulations and because of the unconstrained geometry in HMC after you transform the positive-constrained paraemter. A lognormal goes to zero at zero, as does an inverse gamma. The gamma can go either way depending on the paramers chosen.
A Cauchy(0, 2) can be deceptive because of the wide tails. The central 99% interval is roughly (0.03, 127). Unless you think values of 0.01 and 100 are plausible here, I’d recommend a weakly informative prior (i.e., one determining the rough scale of the answer). Otherwise, unless you have a lot of data, the prior will push the posterior into the tails. Tightening the prior can also help avoid posterior mass piling up near zero and at implausibly high values.