TypeError with arviz.compare

Questions v5
,
1

I’m pretty new to PyMC and ArviZ. Using versions pymc => 4.2.2 and arviz => 0.12.1. I’m exploring arviz.compare. The first argument is a dictionary of InferenceData objects, which coincides with the return type of pymc.sample. However the following simple example blows up with a TypeError. It looks like another post may contain a solution, but my setup is managed by conda, not pip. Any recommendations?

y_obs = stats.norm.rvs(size=100)

# Normal with fixed μ=0 and σ free.
with pm.Model() as model:
    σ = pm.HalfNormal('σ', 1)
    y = pm.Normal('y', 0, sigma=σ, observed=y_obs)
    idata_1 = pm.sample(1000)

# SkewNormal with μ=0, α=1 and σ free.
with pm.Model() as model:
    σ = pm.HalfNormal('σ', 1)
    y = pm.SkewNormal('y', sigma=σ, observed=y_obs)
    idata_2 = pm.sample(1000)

az.compare({'1': idata_1, '2': idata_2})

TypeError                                 Traceback (most recent call last)
File ~/miniconda3/envs/pymc/lib/python3.10/site-packages/pandas/core/indexes/base.py:3803, in Index.get_loc(self, key, method, tolerance)
   3802 try:
-> 3803     return self._engine.get_loc(casted_key)
   3804 except KeyError as err:

File ~/miniconda3/envs/pymc/lib/python3.10/site-packages/pandas/_libs/index.pyx:138, in pandas._libs.index.IndexEngine.get_loc()

File ~/miniconda3/envs/pymc/lib/python3.10/site-packages/pandas/_libs/index.pyx:144, in pandas._libs.index.IndexEngine.get_loc()

TypeError: 'slice(None, None, None)' is an invalid key
...
2

Welcome!

Your code works for me. What version of PyMC/Arivz are you using?

3

Try using conda install arviz==0.14.0 to make sure you are using the latest ArviZ version, you probably have 0.12.1 or something similar installed where that bug was present.

1 Like
4

Awesome, thanks @OriolAbril. I needed to run conda install -c conda-forge arviz, and it auto-updated to the lates version, which is 0.14. That also resolved the issue.

5

I always forget about specifying conda-forge because it is automatic for me, sorry about that. I recommend you install everything from conda forge. I found this tips and tricks page in conda forge docs very helpful.

6

Thanks! That’s helpful, and I’ll update my local configs. My experience is mostly Linux / server, anything Python is usually managed by pip or pipenv, so still learning to navigate conda.

1 Like
7

I am having a different TypeError

_dict, ic, method, b_samples, alpha, seed, scale, var_name)
    177     (ics_dict, scale, ic) = _calculate_ics(compare_dict, scale=scale, ic=ic, var_name=var_name)
    178 except Exception as e:
--> 179     raise e.__class__("Encountered error in ELPD computation of compare.") from e
    180 names = list(ics_dict.keys())
    181 if ic == "loo":

TypeError: Encountered error in ELPD computation of compare.

My models are built with bambi

and my config looks like this:

Python implementation: CPython
Python version       : 3.10.7
IPython version      : 8.9.0

bambi: 0.9.3

numpy     : 1.24.1
lmfit     : 1.1.0
matplotlib: 3.6.3
arviz     : 0.14.0
ipywidgets: 8.0.4
bambi     : 0.9.3
geopandas : 0.11.1
pymc      : 5.0.2
json      : 2.0.9
pandas    : 1.5.3
seaborn   : 0.12.2
plotly    : 5.13.0

Watermark: 2.3.1
1 Like
8

@tcapretto ?

1 Like
9

Make sure you pass idata_kwargs={"log_likelihood": True} when fitting the model. For example:

model.fit(idata_kwargs={"log_likelihood": True})

This is because it’s not computed by default anymore (because PyMC disabled it).

Let me know if that fixes it please :slight_smile:

4 Likes
10

If this doesn’t fix it we’ll need the full traceback to be able to say anything

1 Like
11

I’m having the same error as @Flavio_Codeco_Coelho with another piece of code. Can you please clarify, where would we add this into the original code? Thanks

12

Can you share the piece of code where you’re having the problem? It’s hard for me to tell what you would need to modify if I don’t know how your code works.

With that said, what I’m suggesting in the message you link is to pass the request the log likelihood to be stored in the inference data object when you fit the model. That’s what the model.fit(idata_kwargs={"log_likelihood": True}) does. It should work the same way for you (making sure you put the right variable name in place of model)

13

ch14_practice.py (3.5 KB)
My code is attached, thanks.

The eror is when I run az.compare

TypeError: Encountered error in ELPD computation of compare.

I’m running pymc 5.0.1 and Arviz 0.14.0

14

I think passing idata_kwargs={"log_likelihood": True} to pm.sample() should work

1 Like
15

It did, thanks so much!

2 Likes
16

any logic behind making this change in PyMC?
In my mind, it is one of the main funcionalities of PyMC, to return something that does contain the likelihood for model comparison…
googleing the syntax of idata_kwargs={“log_likelihood”: True} each time I build a model just makes my workflow much more tedious. No real question here, just trying to understand why such changes happen…

17

I think it was changed to avoid some expensive computation or memory usage. I’m not sure. If I recall correctly I once saw @OriolAbril mentioning something about it but I’m not sure.

1 Like
18

It was changed because it can be very costly and memory intensive, and most users are not using these results.

There is an easy way to obtain it after sampling as well: Model comparison — PyMC 5.4.1 documentation

4 Likes
19

thanks! I did not know the pm.compute_log_likelihood- very useful. It is amazing how fast the educational resources for pymc get outdated at pymc gets better-better…

2 Likes