# Plot of mean and HDI is really off

**URL:** <https://discourse.pymc.io/t/plot-of-mean-and-hdi-is-really-off/6031>\
**Category:** Questions\
**Created:** [October 18, 2020, 8:48am UTC](https://discourse.pymc.io/t/plot-of-mean-and-hdi-is-really-off/6031 "2020-10-18T08:48:07Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![saurpan](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/saurpan/32/3042_2.png) [@saurpan](https://discourse.pymc.io/u/saurpan)\
**Post date:** [October 18, 2020, 8:48am UTC](https://discourse.pymc.io/t/plot-of-mean-and-hdi-is-really-off/6031/1 "2020-10-18T08:48:07Z")

</div>

Hi All,

I was practicing pymc3 on spline regression in statistical rethinking chapter 4.  
the plot between the median of a parameter and hdi looks very strange

 ![image](https://canada1.discourse-cdn.com/flex036/uploads/pymc3/original/2X/b/b4393d7f56ae7b698ce3b6a44199e51d8ed0efef.png)

following is my code

```auto
_, ax = plt.subplots(1, 1, figsize=(12, 4))
mu_bar = np.mean(trace4_7['mu'],axis=0)
mu_p = trace4_7["mu"]
ax.plot(d2.year, d2.temp, '.',alpha=0.5)
ax.plot(d2.year, mu_bar)
for i in np.random.randint(0,2000,100):
  ax.plot(d2.year, trace4_7['mu'][i],color='k', alpha=0.1)
pm.plot_hdi(d2.year, y=mu_p,ax=ax)
plt.show()

```

not able to find the problem. looks quite strange. The black curves are randomly sampled and the green shaded graph is plotted by using the plot\_hdi function

By changing the script to

```auto
ax.fill_between(d2.year,az.hdi(mu_p)[:,0],az.hdi(mu_p)[:,1],color="red",alpha=0.5)

```

Now the graph looks ok and closely following the mean value

 ![image](https://canada1.discourse-cdn.com/flex036/uploads/pymc3/original/2X/8/8c54df4671bb096fe85997e706bcbd1096cb8c0f.png)

I am using pymc3 version 3.9.

is this a bug or something?

---

<div class="post-metadata">

**Author:** ![AlexAndorra](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/alexandorra/32/9142_2.png) [@AlexAndorra](https://discourse.pymc.io/u/AlexAndorra)\
**Post date:** [October 19, 2020, 9:35am UTC](https://discourse.pymc.io/t/plot-of-mean-and-hdi-is-really-off/6031/2 "2020-10-19T09:35:16Z")

</div>

Hi @saurpan!  
You can find the PyMC3 port of this chapter [here](https://nbviewer.jupyter.org/github/pymc-devs/resources/blob/master/Rethinking_2/Chp_04.ipynb) – I guess it’ll be useful!

---

<div class="post-metadata">

**Author:** ![saurpan](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/saurpan/32/3042_2.png) [@saurpan](https://discourse.pymc.io/u/saurpan)\
**Post date:** [October 19, 2020, 9:57am UTC](https://discourse.pymc.io/t/plot-of-mean-and-hdi-is-really-off/6031/3 "2020-10-19T09:57:52Z")

</div>

Hi @AlexAndorra thanks for the link. but the issue is the code in that notebook is for different variables and the plot is only the hdi\_plot.  
The puzzling part is when I plot using `pm.plot_hdi()` the plot is different than when I am calculating HDI using `pm.HDI()` and plotting it by using `plt.fill_between() `.  
logically these two should give me the same plot.

---

<div class="post-metadata">

**Author:** ![OriolAbril](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/oriolabril/32/2497_2.png) [@OriolAbril](https://discourse.pymc.io/u/OriolAbril)\
**Post date:** [October 19, 2020, 3:24pm UTC](https://discourse.pymc.io/t/plot-of-mean-and-hdi-is-really-off/6031/4 "2020-10-19T15:24:08Z")

</div>

First of all, I’d recommend you to update to latest ArviZ and PyMC3 if possible, if not please share your ArviZ version too.

I think that the issue here is with the conversion (or no conversion probably) of your trace to `InferenceData`. I’d recommend you first convert your data to `InferenceData` with [`az.from_pymc3`](https://arviz-devs.github.io/arviz/api/generated/arviz.from_pymc3.html#arviz.from_pymc3) ([this blogpost](https://oriolabril.github.io/oriol_unraveled/python/arviz/pymc3/xarray/2020/09/22/pymc3-arviz.html) of mine might help too in addition to the docs). Then use InferenceData or xarray objects to call `hdi` and `plot_hdi`.

My guess with the info available is that `hdi` is converting the trace to inferencedata internally and the automatic conversion works fine -\> the plot then looks fine whereas `plot_hdi` is interpreting the trace as an array and giving nonsensical results.

Side note, keep in mind that the black curves and the hdi you are plotting follow the distrubution of `mu` which I am guessing is the distribution of the _mean of your observations_, they do not follow the distribution of the observations.

---

<div class="post-metadata">

**Author:** ![saurpan](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/saurpan/32/3042_2.png) [@saurpan](https://discourse.pymc.io/u/saurpan)\
**Post date:** [October 20, 2020, 1:55am UTC](https://discourse.pymc.io/t/plot-of-mean-and-hdi-is-really-off/6031/5 "2020-10-20T01:55:32Z")

</div>

Thanks @OriolAbril. I will follow the suggestions and post here.  
Yes you are right I am plotting the mean of my observations.