I found the same issue recently (pymc 4.x). I find out that the computation of the log likelihood is disable on purpose and thus not part of the InferenceData (@ferrine probably knows the reason).
You can still ask pymc to compute the log likelihood for you by doing
Then you can use that idata to compute LOO.
While this will get the values computed I will be careful about the results until getting more info from @ferrine or others knowing why it is currently disable by default.
Thank you very much for the reply. I’ll check it out and see if my results make sense, although I will not make any decision until we get more information.
I think the comparison is not fair. Consider the case when your approximate posterior converged to have narrow high density region, while the true posterior has a broader shape. In the comparison, the narrow model would always win which is not true. I’m not against of returning the log likelihood, but this leads to confusion in interpretation
@ferrine If I understand it correctly this paper argues for using PSIS loo cross validation for identifying problems with ADVI approximations. They cite the R loo package, but I believe this is the same as the arviz.loo, which in turn uses the log_likelihood component from the InferenceData.