# Bounded variable logp

**URL:** https://discourse.pymc.io/t/bounded-variable-logp/1708
**Category:** Questions
**Created:** [August 12, 2018, 12:57pm UTC](https://discourse.pymc.io/t/bounded-variable-logp/1708 "2018-08-12T12:57:16Z")
**Posts on this page:** 7
**Page:** 1

<div class="post-metadata">

### Author: ![pjo](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/pjo/32/1046_2.png) [@pjo](https://discourse.pymc.io/u/pjo)
#### Post date: [August 12, 2018, 12:57pm UTC](https://discourse.pymc.io/t/bounded-variable-logp/1708/1 "2018-08-12T12:57:16Z")

</div>

My model uses several bounded uniform distributions, which I’m representing with bounded normal distributions with large, uninformative standard deviations.

When I was debugging my model, I tried to verify that I was implementing these pseudo-uniform variables correctly by checking the logp of them. However, the logp of these variables falls off asymptotically as it approach the variable bounds.

I did some tests with simple models and they work just fine, so I must not understand how the bounded variables are being handled. Why does their logp fall off so drastically near their bounds?

```
Y = np.array([0])

model = pm.Model()
with model:
    alpha = pm.Bound(pm.Normal, lower=0, upper=10)('alpha', mu=0., sd=100.)
    logp = pm.Deterministic('logp', model.logpt)
    Y_obs = pm.Normal('Y_obs', mu=alpha, sd=1, observed=Y)

plt.xlabel('alpha')
plt.ylabel('logp')
plt.scatter([_['alpha'] for _ in trace],[_['lp'] for _ in trace]);

```

![download](https://canada1.discourse-cdn.com/flex036/uploads/pymc3/original/2X/3/3c82341fdfc066cf925236d08d292c4df77e06df.png)

---

<div class="post-metadata">

### Author: ![junpenglao](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/junpenglao/32/8_2.png) [@junpenglao](https://discourse.pymc.io/u/junpenglao)
#### Post date: [August 12, 2018, 1:08pm UTC](https://discourse.pymc.io/t/bounded-variable-logp/1708/2 "2018-08-12T13:08:06Z")

</div>

> [@pjo](#):
>
> However, the logp of these variables falls off asymptotically as it approach the variable bounds.

What do you mean? You mean much fewer samples at the bound?

---

<div class="post-metadata">

### Author: ![pjo](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/pjo/32/1046_2.png) [@pjo](https://discourse.pymc.io/u/pjo)
#### Post date: [August 12, 2018, 1:24pm UTC](https://discourse.pymc.io/t/bounded-variable-logp/1708/3 "2018-08-12T13:24:22Z")

</div>

It seems like the sampler is handling the bounds fine - I get plenty of samples near the bounds - but I don’t understand why the logp falls off close to the variable bounds (shown in the attached image).

---

<div class="post-metadata">

### Author: ![junpenglao](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/junpenglao/32/8_2.png) [@junpenglao](https://discourse.pymc.io/u/junpenglao)
#### Post date: [August 12, 2018, 1:26pm UTC](https://discourse.pymc.io/t/bounded-variable-logp/1708/4 "2018-08-12T13:26:40Z")

</div>

I dont quite get what do you mean by “falls off”.

---

<div class="post-metadata">

### Author: ![pjo](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/pjo/32/1046_2.png) [@pjo](https://discourse.pymc.io/u/pjo)
#### Post date: [August 12, 2018, 1:31pm UTC](https://discourse.pymc.io/t/bounded-variable-logp/1708/5 "2018-08-12T13:31:12Z")

</div>

The logp becomes more negative close to the bounds. I would expect it to be flat through the entire interval.

---

<div class="post-metadata">

### Author: ![junpenglao](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/junpenglao/32/8_2.png) [@junpenglao](https://discourse.pymc.io/u/junpenglao)
#### Post date: [August 12, 2018, 2:37pm UTC](https://discourse.pymc.io/t/bounded-variable-logp/1708/6 "2018-08-12T14:37:50Z")

</div>

Oh right I get what you mean now. That’s due to the [auto-transformation of bounded variables](http://docs.pymc.io/notebooks/api_quickstart.html#Automatic-transforms-of-bounded-RVs) in pymc3:

You can play around with it by turning off the `transform` kwarg (using uniform here as demonstration):

```python
with pm.Model() as model:
# alpha = pm.Bound(pm.Normal, lower=0, upper=10)('alpha', mu=0., sd=100., transform=None)
    alpha = pm.Uniform('alpha', 0, 10, transform=None)
logp = model.logp
x_ = np.linspace(0., 10., 1000)
plt.plot(x_, np.exp([logp(dict(alpha=x)) for x in x_]));

```

with  
 ![uniform](https://canada1.discourse-cdn.com/flex036/uploads/pymc3/original/2X/8/8a07cb47f3d778bb4e45a99305d8fc91151746a9.png)  
So no surprise here.

But that’s not the sampler is “seeing”, as we prefer to operate in the unbounded space which makes sampling and approximation much easier:

```python
with pm.Model() as model:
# alpha = pm.Bound(pm.Normal, lower=0, upper=10)('alpha', mu=0., sd=100.)
    alpha = pm.Uniform('alpha', 0, 10)
logp = model.logp
x_ = np.linspace(1e-10, 10.-1e-10, 1000) # avoid -inf and inf at the bound
x_2 = alpha.transformation.forward_val(x_)
plt.plot(x_2, np.exp([logp(dict(alpha_interval__=x)) for x in x_2]));

```

![uniform%20in%20transformed%20space](https://canada1.discourse-cdn.com/flex036/uploads/pymc3/original/2X/0/0433873a6f1d8f302bd5c7363dff004fb14ce58a.png)

Unfortuantely turning the above figure back to uniform is not that easy in PyMC3, as we dont have forward jacobian implemented. You can give it a try as exercise 😉

---

<div class="post-metadata">

### Author: ![pjo](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/pjo/32/1046_2.png) [@pjo](https://discourse.pymc.io/u/pjo)
#### Post date: [August 15, 2018, 1:00pm UTC](https://discourse.pymc.io/t/bounded-variable-logp/1708/7 "2018-08-15T13:00:39Z")

</div>

Thanks for the answer, @junpenglao this is very helpful.
