# Truncated Normal

**URL:** <https://discourse.pymc.io/t/truncated-normal/6338>\
**Category:** Questions\
**Created:** [December 1, 2020, 6:13pm UTC](https://discourse.pymc.io/t/truncated-normal/6338 "2020-12-01T18:13:16Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![BjornHartmann](https://avatars.discourse-cdn.com/v4/letter/b/a87d85/32.png) [@BjornHartmann](https://discourse.pymc.io/u/BjornHartmann)\
**Post date:** [December 1, 2020, 6:13pm UTC](https://discourse.pymc.io/t/truncated-normal/6338/1 "2020-12-01T18:13:16Z")

</div>

hi!  
I am having trouble with truncatednormal distributions. In my understanding, these are just normal distributions with a specified restriction on upper and lower bounds.

I however get divergencies when using them, even on the simplest case (code below). Can anyone help me explain this?

```
import pymc3 as pm

import matplotlib.pyplot as plt

with pm.Model() as model:

    N = pm.TruncatedNormal('N', mu=0,sd=5,lower=0)

    trace = pm.sample()

    pm.traceplot(trace)

    plt.savefig("trunc_test")

    plt.close()
```

---

<div class="post-metadata">

**Author:** ![luisroque](https://avatars.discourse-cdn.com/v4/letter/l/7c8e57/32.png) [@luisroque](https://discourse.pymc.io/u/luisroque)\
**Post date:** [December 2, 2020, 10:07am UTC](https://discourse.pymc.io/t/truncated-normal/6338/2 "2020-12-02T10:07:53Z")

</div>

I believe that it is because you are giving it a Normal distribution with mean 0 and excluding all the mass below zero, which makes it hard for the sampler.

Try the following:

```auto
y = np.random.normal(0.1,1,size=100)

with pm.Model() as model:

    N = pm.TruncatedNormal('N', mu=0,sd=5,lower=0)
    
    y_pred = pm.Normal('y_pred', N, 1, observed = y)

    trace = pm.sample()

    pm.traceplot(trace)

```

We get divergencies. Notice that the mean is extremely close to the bound. If we get a little further away from those extremes:

```auto
y = np.random.normal(0.5,1,size=100)

with pm.Model() as model:

    N = pm.TruncatedNormal('N', mu=0,sd=5,lower=0)
    
    y_pred = pm.Normal('y_pred', N, 1, observed = y)

    trace = pm.sample()

    pm.traceplot(trace)

```

No divergencies.

Hope it helps!

---

<div class="post-metadata">

**Author:** ![BjornHartmann](https://avatars.discourse-cdn.com/v4/letter/b/a87d85/32.png) [@BjornHartmann](https://discourse.pymc.io/u/BjornHartmann)\
**Post date:** [December 2, 2020, 1:40pm UTC](https://discourse.pymc.io/t/truncated-normal/6338/3 "2020-12-02T13:40:12Z")

</div>

hi, @luisroque,  
Thank you for getting back to me!  
I am having trouble accepting that solution, as restricting the truncatednormal upper and lower bounds even more still works as long as i also specify an _upper_-bound.

as an example, this gives no divergencies:  
`N = pm.TruncatedNormal('N', mu=0,sd=5,lower=3,upper=8)`  
while this does:  
`N = pm.TruncatedNormal('N', mu=0,sd=5,lower=3)`

---

<div class="post-metadata">

**Author:** ![BjornHartmann](https://avatars.discourse-cdn.com/v4/letter/b/a87d85/32.png) [@BjornHartmann](https://discourse.pymc.io/u/BjornHartmann)\
**Post date:** [December 2, 2020, 2:27pm UTC](https://discourse.pymc.io/t/truncated-normal/6338/4 "2020-12-02T14:27:39Z")

</div>

Just wanted to add that i see the same with bounded variables with pm.Bound.  
Could this be a bug?

---

<div class="post-metadata">

**Author:** ![luisroque](https://avatars.discourse-cdn.com/v4/letter/l/7c8e57/32.png) [@luisroque](https://discourse.pymc.io/u/luisroque)\
**Post date:** [December 3, 2020, 11:12am UTC](https://discourse.pymc.io/t/truncated-normal/6338/5 "2020-12-03T11:12:26Z")

</div>

Hmm, you are right and I can’t understand why. Would be good if someone with more experience could help. I will leave some notes from some digging that I did.

From what I see on the code, `PyMC3` transforms the variables when you define a `TruncatedNormal`. I can also see that the logp is increased proportionally in the region that you are considering, which results in a higher density. When you use both lower and upper bounds, naturally you are increasing it from both sides. I would imagine that it could help the sampler when exploring the space.

I can see that if we increase the `target_accept` with just the lower bound considered we can manage to get no divergencies but our effective sample size is quite low (it is low nevertheless).

I can see that with a very high upper bound the behaviour is the expected, i.e. the same when not defining one. The strange thing is in between, like when using a sigma of 10 and an upper bound of 100. I don’t see why we get no divergencies and a significantly bigger ess.

---

<div class="post-metadata">

**Author:** ![BjornHartmann](https://avatars.discourse-cdn.com/v4/letter/b/a87d85/32.png) [@BjornHartmann](https://discourse.pymc.io/u/BjornHartmann)\
**Post date:** [December 3, 2020, 3:27pm UTC](https://discourse.pymc.io/t/truncated-normal/6338/6 "2020-12-03T15:27:07Z")

</div>

yes, thank you,  
would be great to figure this out!

---

<div class="post-metadata">

**Author:** ![BjornHartmann](https://avatars.discourse-cdn.com/v4/letter/b/a87d85/32.png) [@BjornHartmann](https://discourse.pymc.io/u/BjornHartmann)\
**Post date:** [December 4, 2020, 2:58pm UTC](https://discourse.pymc.io/t/truncated-normal/6338/7 "2020-12-04T14:58:57Z")

</div>

does anyone have any suggestions on how to understand this? 😃

---

<div class="post-metadata">

**Author:** ![BjornHartmann](https://avatars.discourse-cdn.com/v4/letter/b/a87d85/32.png) [@BjornHartmann](https://discourse.pymc.io/u/BjornHartmann)\
**Post date:** [December 8, 2020, 12:19pm UTC](https://discourse.pymc.io/t/truncated-normal/6338/8 "2020-12-08T12:19:06Z")

</div>

Hi!  
I have still not figured this out, and i believe there is a simple solution to the problem. Still hoping for a suggestion on how to explain this!

Thanks

---

<div class="post-metadata">

**Author:** ![ricardoV94](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/ricardov94/32/5775_2.png) [@ricardoV94](https://discourse.pymc.io/u/ricardoV94)\
**Post date:** [January 29, 2021, 2:56pm UTC](https://discourse.pymc.io/t/truncated-normal/6338/9 "2021-01-29T14:56:56Z")

</div>

I also get divergence problems when sampling from a HalfNormal

```python
with pm.Model() as m: 
  x = pm.HalfNormal('x', 1) 
  trace = pm.sample() 
  # Divergences

```

I think NUTS is struggling to sample because the peak of the distribution is at the very boundary of the domain. The problems seem to go away if the mean is enough away from the boundary:

```auto
with pm.Model() as m: 
  x = pm.TruncatedNormal('x', lower=0, mu=3, sigma=1) 
  trace = pm.sample() 
  # No Divergences

```

If you want NUTS to sample you might want to give it an extra legroom by tweaking `tune` and `target_accept`:

```python
with pm.Model() as m: 
  x = pm.TruncatedNormal('x', lower=0, mu=0, sd=5) 
  trace = pm.sample(tune=2000, target_accept=.95) 
  # No divergences

```

If anyone else can chime in and confirm this is reasonable behavior from NUTS, that would be really helpful.

This might be related: [Zero-excluding priors are probably a bad idea for hierarchical variance parameters « Statistical Modeling, Causal Inference, and Social Science](https://statmodeling.stat.columbia.edu/2018/05/04/zero-excluding-priors-probably-bad-idea-hierarchical-variance-parameters/)