I recently compared some models using pm.compare. When I use “stacking”, the rankings by LOO don’t match those by weight: index rank loo p_loo d_loo weight se dse warning loo_scale 0 Full 0 -1032.49 58.5355 0 0.785312 19.6743 0 1 log 1 Decay only 1 -1039.55 53.1845 7.06546 0.127025 21.19…

Stacking weights do not necessarily follow the same order as loo/waic order nor pseudo-BMA weigths. There is a very clear example on this in the loo R package vignettes, in " Example: Oceanic tool complexity" section . The results are the following: waic_wts pbma_wts pbma_BB_wts stacking_wts…

Bayesian model averaging: ranking of model weights and LOO don't match

Questions

AlexAndorra March 10, 2020, 9:28pm 8

Super clear – and enlightening, thank you Oriol!

Topic		Replies	Views
Problem in model comparison	3	237	February 8, 2024
Arviz compare: rank inconsistent with weight v5 arviz	2	606	June 24, 2022
Comparing models and selecting it with "weight" of arviz.compare Questions	1	1005	June 10, 2021
Comparing models - ranks vs weights	2	55	July 30, 2024
WAIC & LOO scaling Questions	1	558	April 7, 2020

Bayesian model averaging: ranking of model weights and LOO don't match

Related topics