I am looking at a place where I’m getting a bad initial energy, and I have found the term (thanks to everyone for the trick of looking at the BasicRVs).
There’s a node (InverseGamma, the variance of a population that I observe), that has what looks like a normal value as observation (it’s about 0.56, which is reasonable), but that has a -inf energy.
The model is generated semi-automatically, so I have looked into all the RVs’ distributions, which so far look reasonable. But when I look at the critical random variable’s distribution, all I see is this:
model['sigma_Input00 replicate 0'].distribution.alpha
Out[21]: Elemwise{true_div,no_inplace}.0
Is there any way to print this more readably so that I can tell if something bad is happening here? I’ve parameterized these variables using means and standard deviations, instead of the standard alpha and beta,
Ah, I see that I can answer my own question. The function I needed was theano.printing.pprint() as in:
In : theano.printing.pprint(model['sigma_Input00 replicate 0'].distribution.alpha)
Out[21]: '(((TensorConstant{2} * (hyper_sigma_mu_Input00 * hyper_sigma_mu_Input00)) + (ViewOp(exp(hyper_sigma_sigma_Input00_log__)) * ViewOp(exp(hyper_sigma_sigma_Input00_log__)))) / (hyper_sigma_mu_Input00 * hyper_sigma_mu_Input00))'
This shows how PyMC3 is taking the parameters I passed as mu and sd and using them to compute alpha.
I’m still not sure how to deiagnose my -inf logp, but I will try to ask more focused questions later.
Thanks. I poked around painfully and found… a hyper-parameter for σ that was a Normal instead of a TruncatedNormal, yielding occasionally a negative value and causing the error.
Here’s a follow-up question: I have done this often enough that I’d love a crutch to help me avoid the same mistake. Is there some way to attach to a random variable something like a contract so that (as in this case), if a sampled value violates the constraint (e.g., falls below zero), then I can throw an error message that is more helpful than just a bad energy function? E.g., be able to say “Standard deviation variable X is less than zero.” This isn’t something PyMC3 can know, but the modeler could indicate that some variable is, for example, used as a standard deviation or variance.
In general, it seems that checks like that, during sampling, could provide a quicker pointer to model repair. But I’m not sure where in the code I might look to place such checks. Thanks!
Well, we have some check but for parameter domain but they dont work on tensors, so yeah not great. In general, unless we have type check it seems a bit difficult to cover all use case (e.g., parameter being output from some function), but we should definitively improve this - suggestion welcome!