Custom log likelihood and LOO

Hi there,

If I have a model with a custom likelihood function defined with a pm.Potential combining some distributions with logp values, the returned trace seems to miss the log_likelihood attribute and az.loo complains. Can the solution be to make the likelihood values available as a pm.Deterministic and make it an attribute?

Cheers,
MV

1 Like

Hi,

It is possible to use any log likelihood array to compute loo, it doesn’t need to be automatically retrieved when converting to InferenceData. There are some cases where getting pointwise log likelihood values automatically is not possible, there are other cases where getting the pointwise values automatically is possible but then it’s not clear how to compute loo from there.

Luckily, az.loo takes the data from the log_likelihood group of the input inferencedata, and uses this to estimate elpd_loo. You should therefore be able to add a deterministic (which will be stored in the posterior) and then “move” it to the log_likelihood group. Another option is to compute that by hand afterwards with numpy and xarray (not too different from computing by hand in a deterministic). There is an example on hand generated log likelihood data in Refitting PyMC3 models with ArviZ (and xarray) — ArviZ dev documentation

I would like to note however, that defining and/or choosing what are your pointwise log likelihood values may not be obvious, there could even be multiple correct ways of doing so, the only difference between them being the question they answer. Take a look at this notebook for example

1 Like

Thanks for you reply and for the links, that’s exactly what I needed to go forward!