Model comparison of hierarchical models

Questions version agnostic
1

Hi,

I am trying to perform a model comparison for two hierarchical models.
I was wondering if there’s any way to compare the models at the subject level, i.e. for subject 1 model 1 is better, for subject 10 model 2 is better, etc… as I would be interested in quantifying which model fits the subjects best (right now I’m using the weight value returned by az.compare).
In the past, I used to compute AIC and BIC for each subject/model and then check for the lowest AIC/BIC for each subject/model.
Can something like this be done/does it make sense in a Bayesian setting?
Thanks!
Filippo

2

WAIC or LOO would be the way to go. AIC and BIC are not applicable in cases where you are using a Bayesian model + priors + MCMC.

3

Thanks!
Using the loo_i/waic_i outputs from pointwise LOO/WAIC should be exactly what I need for subject-level comparisons.

1 Like