This might be a overly simple question, but what does it mean when the displayed Average Loss is negative, e.g.:
Average Loss = -1.7241e+05: 74%|███████▎ | 7366/10000 [24:12<08:39, 5.07it/s]
As far as I understand it represents the negation of the ELBO, which should always be negative. Does this indicate some error in my model/set-up?
Thanks!
The displayed Loss value does not bare much meaning - some constant might be drop during the model set up. As long as the number is getting smaller you dont need to worry too much about it.
Great, thank you very much!
Is a -inf loss similarly indicative of problems as positive inf?
Yes if you are seeing them consistently. It happens sometimes as the monte carlo samples from the ADVI could make the elbo goes to inf, but as training going further it should not appear any more