Numerical Issues with StickBreaking in ADVI

Softmax is something like softmax = lambda x: exp(x) / sum(exp(x)). Every time there is some kind of self normalization like dividing the sum it usually makes your model unidentifiable. You might not see it plotting the transformed variable decomp, but it is likely visible in the raw space decomp_stickbreaking__

1 Like