Sampling Bernoulli + Exponential cannot reproduce original distribution

Yes. Although you should note that your model implies that your zero expenses are only roughly zero and your prior suggests that the “non-zero” expenses can reasonably be expected to be negative (or positive with equal probability). But if that isn’t too far off of your actual application, it might be fine. But if your posterior admits negative “non-zero” expenses, things could get a bit weird in a predictive context. Just something to be aware of.