@OriolAbril Thanks for the reply. I assume you are talking about the first line here:
def _loo_pit(y, y_hat, log_weights):
"""Compute LOO-PIT values."""
sel = y_hat <= y
if np.sum(sel) > 0:
value = np.exp(_logsumexp(log_weights[sel]))
return min(1, value)
else:
return 0
where y_hat are the samples and y the observed value. Indeed looking at the the list of loo pit values shows one sees an excess of values equal to 1. These seem to result from cases where sel = y_hat <= y being satisfied by all data points for a given observation and corresponding samples because y_hat == y is satisfied for almost all of them. So these are samples which are (almost) completely (right I guess?) censored. On the other hand if you change this to sel = y_hat < y then very few data points satisfy this and you get an excess of near-zero values. Would it fair to try to exclude observations which are censored from such analysis or is there are a better way to improve the loo-pit analysis to account for such values?