Hi,
I am using minibatch ADVI to fit models for my use case, but for some reason I often very strange convergence/hist plots like this:
The elbo seems to stabilize at 30000, be stable for a while and then starts to get very unstable from around 60000.
I would like to know if you can point me to generic reasons that can cause such hist plots and what I could do to tackle this problem (I am new to the world of Bayesian modeling).
I would prefer not to show my model, as I got this problem with different models, so I was thinking the problem must be something more generic.
I am using hierarchical models and one thing that I saw is that when certain groups only have very small samples (1 or 2) they tend to be estimated very wrong.