In sample deviance - better or worse than mean estimate of IC?

Questions
1

I’m running the following:

model_compare = az.compare({'Centered 8 schools': az.load_arviz_data('centered_eight'),
                 'Non-centered 8 schools': az.load_arviz_data('non_centered_eight')}, scale='log')
az.plot_compare(model_compare)

and

model_compare = az.compare({'Centered 8 schools': az.load_arviz_data('centered_eight'),
                 'Non-centered 8 schools': az.load_arviz_data('non_centered_eight')}, scale='deviance')
az.plot_compare(model_compare)

The output looks as:

image
image906×598 32.1 KB

If we use ‘log’ scale, then higher values are better, while if we use ‘deviance’, then lower values are better. What I don’t understand is that, in the ‘log’ scale case, the in-sample deviance is “worse” than the mean estimate (i.e. it’s to its left), while in the ‘deviance’ scale the in-sample deviance is “better” than the mean estimate (i.e. it’s to its left).

I would’ve expected the in-sample deviance to always be “better” than the mean estimate of the IC, because it doesn’t include penalisation given by the number of parameters. I’m also surprised that the scale determines whether it’s “better” or “worse” than the mean IC.

2

Looks like this is fixed on ‘master’