# Why are models hierarchical?

**URL:** <https://discourse.pymc.io/t/why-are-models-hierarchical/7413>\
**Category:** Questions\
**Created:** [May 8, 2021, 11:45am UTC](https://discourse.pymc.io/t/why-are-models-hierarchical/7413 "2021-05-08T11:45:24Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![John\_Mubarak](https://avatars.discourse-cdn.com/v4/letter/j/e56c9b/32.png) [@John\_Mubarak](https://discourse.pymc.io/u/John_Mubarak)\
**Post date:** [May 8, 2021, 11:45am UTC](https://discourse.pymc.io/t/why-are-models-hierarchical/7413/1 "2021-05-08T11:45:24Z")

</div>

Hi!  
I’m starting again with PyMC3, and I was looking for a simple and intuitive answer to a doubt that has arisen in my re-study:

What are the reasons for hierarchizing the models, and what are the advantages?

Sorry if the question is too simple.  
Thaks!

---

<div class="post-metadata">

**Author:** ![ricardoV94](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/ricardov94/32/5775_2.png) [@ricardoV94](https://discourse.pymc.io/u/ricardoV94)\
**Post date:** [May 8, 2021, 3:54pm UTC](https://discourse.pymc.io/t/why-are-models-hierarchical/7413/2 "2021-05-08T15:54:37Z")

</div>

It’s a very broad question. Here is a very simplistic take:

In general hierarchical models are used because you assume that one measurement (can) provide information for another measurement, even if they are not necessarily causally linked. For instance, knowing that five instances of variable `X` have a mean of 5, and a standard deviation of 2, can inform you about the expected values of the next observations of `X`, even if you have no particular information about how the instances of `X` are related to each other.

In contrast, non-hierarchical models, start from “scratch” when doing inferences about each instance of `X`. This means you need more data to reduce your uncertainty to the same extent that a hierarchical model could.

---

<div class="post-metadata">

**Author:** ![jonsedar](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/jonsedar/32/2590_2.png) [@jonsedar](https://discourse.pymc.io/u/jonsedar)\
**Post date:** [May 9, 2021, 3:37am UTC](https://discourse.pymc.io/t/why-are-models-hierarchical/7413/3 "2021-05-09T03:37:38Z")

</div>

I’d add that hierarchical models can helpfully let you express more of your knowledge about the data generating process. This can get quite elaborate and tbh is where I initially learned the principle. E.g the classic model of determining whether a coin flip is fair and including in the model a nested hierarchy of details about where the coin was minted (assembly line, factory, region) etc.

To @ricardoV94’s excellent point about constraining variance, I think this is a very cool and subtle property of hierarchical models and worth a little more explanation, so I’ll try.

Imagine you have a really simple linear model `y ~ N(a + bx, e)`. You have a categorical (aka factor) predictor variable `v` with `h` levels that you haven’t yet included. Where can you add `v`?

1. If (as my notation suggests) you currently multiply coefficient `a` by `1`, then this is a simple _pooled_ intercept that already contains some information about `v`. However that information is also mixed up with the rest of the model, so it’s not particularly useful.
2. You could choose to place `v` on the intercept as `a[h]`, which gives you an _unpooled_ intercept: a separate intercept for each of `h` levels. (FYI this is the same as one-hot-encoding `v` into several new binary features).
3. I’ll skip anything more elaborate for now

So now your model tells you something about how each level `h` of variable `v` correlates with `y`, which is great. However (and to elaborate on @ricardoV94’s point) you might have an imbalance of factor-values in `v`: e.g. level `h_1` is present in 1% of the observations, `h_2` in 29%, and `h_3` is present in 70% of the observations. In this _unpooled_ model, the coefficients for `a_1`, `a_2`, `a_3` are fitted completely independently and so the coefficient for `a_1` will be much weaker (have more variance) than `a_2` and `a_3`.

In reality this factor-value imbalance might be misleading because the factors aren’t necessarily orthogonal: there might be some shared information amongst them, and we could try to constrain the variance in the under-observed factor-value by using the other factor values.

To do this we could introduce a hierarchy onto this intercept to achieve _partial pooling_: `a ~ N(w, 1)`, `a[h]`. Now `w` will fit to a balanced mean of the levels `h_*`, and each coefficient `a_*` will _shrink_ closer towards `w`, naturally based on their proportions in the data. I.e. `a_2` will move a little closer to `a_1` and `a_3` will move a lot closer to `a_1`. The variance in `a_2` and especially in `a_1` would also reduce, giving us a more robust estimate or ‘sharing power’ between the factor-value levels.

As always in this game, there’s no hard and fast rules about building these hierarchies, or even if they’re worthwhile for your particular dataset, and they can have very strange effects on the joint posterior and thus the sampling. In some circumstances it’s a really powerful tool.

---

<div class="post-metadata">

**Author:** ![wardera](https://avatars.discourse-cdn.com/v4/letter/w/4da419/32.png) [@wardera](https://discourse.pymc.io/u/wardera)\
**Post date:** [May 11, 2021, 8:23pm UTC](https://discourse.pymc.io/t/why-are-models-hierarchical/7413/4 "2021-05-11T20:23:07Z")

</div>

I’m not as smart as these other guys, but here is my simple rule. If your data is clustered, then it can make sense because errors can be correlated. So it helps avoid violating assumptions of other statistical methods. Also, the shrinkage towards the mean can help avoid overly confident predictions.

---

<div class="post-metadata">

**Author:** ![garciascms](https://avatars.discourse-cdn.com/v4/letter/g/ecc23a/32.png) [@garciascms](https://discourse.pymc.io/u/garciascms)\
**Post date:** [May 12, 2021, 7:41am UTC](https://discourse.pymc.io/t/why-are-models-hierarchical/7413/5 "2021-05-12T07:41:59Z")

</div>

Hi guys, I am also new to PyMC3, thanks for the thorough explanation! I have an additional question about models hierarchical. Let’s say I am trying to find the probability of occurence of demand values for multiple retail products. In the dataset, each product has attributes such as color, brand and whatnot. Like, I want to make sure that characteristics of the products are used in the model to find the prediction for all the products. Is it what you mean with:

" one measurement (can) provide information for another measurement, even if they are not necessarily causally linked."

I believe an example would help. Let’s say I want probabilistic forecast of demand for two shirts, one of color blue, and one of color red. The demand probability for one tshirt should be linked to the demand for the other shirt. Is this achievable with PyMC3?

Thank you!

---

<div class="post-metadata">

**Author:** ![jonsedar](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/jonsedar/32/2590_2.png) [@jonsedar](https://discourse.pymc.io/u/jonsedar)\
**Post date:** [May 12, 2021, 9:59am UTC](https://discourse.pymc.io/t/why-are-models-hierarchical/7413/6 "2021-05-12T09:59:55Z")

</div>

Sure, perhaps the examples notebooks will help: there’s a few on hierarchical modelling:

- [GLM: Hierarchical Linear Regression — PyMC3 3.11.2 documentation](https://docs.pymc.io/pymc-examples/examples/generalized_linear_models/GLM-hierarchical.html)
- [Hierarchical Partial Pooling — PyMC3 3.11.2 documentation](https://docs.pymc.io/pymc-examples/examples/case_studies/hierarchical_partial_pooling.html)
- [A Hierarchical model for Rugby prediction — PyMC3 3.11.2 documentation](https://docs.pymc.io/pymc-examples/examples/case_studies/rugby_analytics.html)

Likely the core snippet you want to think about is something like:

```auto
...
    beta_mu= pm.Normal('beta_mu', mu=0, sigma=1)
    beta = pm.Normal('beta', mu=beta_mu, sigma=1, shape=n_colors)
....

```