I am comparing a set of mixed models, sampled with PyMC 4.0 with a few predictors and slight changes in the random slopes/intercept structures between the models. I compare the output with az.compare (with both loo and waic). My first ranked model has the highest weight, but my 3ranked model has…

Yes, this can happen and it is not that uncommon. See Bayesian model averaging: ranking of model weights and LOO don't match for more detailed answer

Arviz compare: rank inconsistent with weight

jjarato June 24, 2022, 10:25am 3

Thanks!

Topic		Replies	Views
Bayesian model averaging: ranking of model weights and LOO don't match Questions	10	1360	March 14, 2020
Comparing models and selecting it with "weight" of arviz.compare Questions	1	1005	June 10, 2021
Problem in model comparison	3	237	February 8, 2024
Comparing models - ranks vs weights	2	55	July 30, 2024
WAIC & LOO scaling Questions	1	558	April 7, 2020