I am comparing a set of mixed models, sampled with PyMC 4.0 with a few predictors and slight changes in the random slopes/intercept structures between the models. I compare the output with az.compare (with both loo and waic). My first ranked model has the highest weight, but my 3ranked model has…

Arviz compare: rank inconsistent with weight

OriolAbril June 23, 2022, 8:24pm 2

Yes, this can happen and it is not that uncommon. See Bayesian model averaging: ranking of model weights and LOO don't match for more detailed answer

Topic		Replies	Views
Problem in model comparison	3	289	February 8, 2024
Comparing models - ranks vs weights	2	100	July 30, 2024
Comparing models and selecting it with "weight" of arviz.compare Questions	1	1053	June 10, 2021
Bayesian model averaging: ranking of model weights and LOO don't match Questions	10	1495	March 14, 2020
Is model with high or low WAIC and LOO better version agnostic arviz	1	1713	August 12, 2021