Average loss for ADVI never decrease for my model

I would describe what you are seeing as the model and the data and the sampler not getting along very nicely. Uniform distributions tend to be tough with the NUTS sampler, for example. And the fact that you are incorporating random elements into your model (e.g., the calls to np.random.normal) is generally not a good idea in the context of a PyMC model.

Instead of debugging your current model, I would suggest stepping back and describing what you are trying to do. Once you and others here have a handle on that, you can do as @twiecki suggest and start with a simple (possibly too-simple) version of the model and gradually add pieces to it so that you know when (if) sampling starts to be problematic.

1 Like