I am looking at the SVM example in PyMC. When I fit this model with ADVI, the loss becomes negative. What is wrong with this model? I don’t really understand.
All the code comes from that example, and I only added the ADVI part myself.
I have checked the post, but it doesn’t seem to be the same case.
That model diverges to -inf.
Negative loss is fine. pm.fit minimizes the negative ELBO loss, which takes values on \mathbb R. As long as its not -inf, you’re good.
Better to check the fit on the data than to try to read the loss function tea leaves.
1 Like