# Linear Regression - Difference between these two models?

**URL:** <https://discourse.pymc.io/t/linear-regression-difference-between-these-two-models/2487>\
**Category:** Questions\
**Created:** [January 7, 2019, 12:09pm UTC](https://discourse.pymc.io/t/linear-regression-difference-between-these-two-models/2487 "2019-01-07T12:09:05Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Paras\_Chopra](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/paras_chopra/32/1385_2.png) [@Paras\_Chopra](https://discourse.pymc.io/u/Paras_Chopra)\
**Post date:** [January 7, 2019, 12:09pm UTC](https://discourse.pymc.io/t/linear-regression-difference-between-these-two-models/2487/1 "2019-01-07T12:09:06Z")

</div>

Hello,

I’m doing linear regression and I’m wondering which is the correct / better way of observing independent variable (weight in my case, the dependent variable is height).

One way is to model independent variable as a (normal) distribution and conditioning on observed data:

m = pm.Model()

with m:

```
alpha = pm.Uniform('alpha', lower=0, upper=300)
beta = pm.Normal('beta', mu=0, sd=50)
sigma_height = pm.Uniform('sigma_height', lower=0, upper=50)

weight = pm.Normal('weight', mu=np.mean(d2['weight']), sd=np.std(d2['weight']), observed=d2['weight'])
mu_height = alpha + beta * (weight-np.mean(d2['weight']))

height = pm.Normal('height', mu=mu_height, sd=sigma_height, observed=d2['height'])

```

But I discovered that I can simply add weight directly from dataframe (without first modeling it as a normal distribution)

m = pm.Model()

with m:

```
alpha = pm.Uniform('alpha', lower=0, upper=300)
beta = pm.Normal('beta', mu=0, sd=50)
sigma_height = pm.Uniform('sigma_height', lower=0, upper=50)

mu_height = alpha + beta * (d2['weight']-np.mean(d2['weight']))

height = pm.Normal('height', mu=mu_height, sd=sigma_height, observed=d2['height'])

```

My questions are:

- How are these two models treated by PyMC3 during MCMC inference? What’s the difference? (Specifically for the second one, since dataframe (d2[‘weight’]) has a list of values, how is mu\_height calculated (since it is scalar)? During sampling, Is dataframe also sampled (uniformly?) to arrive at mu\_height?
- Which one should I be using for regression? Are there any pros and cons of each approach?

By the way, both of the approaches give similar estimations for unknowns (alpha, beta, sigma\_height).

---

<div class="post-metadata">

**Author:** ![junpenglao](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/junpenglao/32/8_2.png) [@junpenglao](https://discourse.pymc.io/u/junpenglao)\
**Post date:** [January 7, 2019, 3:39pm UTC](https://discourse.pymc.io/t/linear-regression-difference-between-these-two-models/2487/2 "2019-01-07T15:39:06Z")

</div>

The two model will be treated the same in MCMC, because

```python
weight = pm.Normal('weight', mu=np.mean(d2['weight']), sd=np.std(d2['weight']), observed=d2['weight'])

```

Adds a constant to the model logp, thus has not effect during MCMC. Moreover, `weight` will be a tensor with value set to `d2['weight']`.

You should use the first one, because the second one does not add any new information and could be confusing. But if you have uncertainty in the covariates you want to model you can also do:

```python
mu_weight = pm.Normal(...)
sd_weight = pm.HalfNormal(...)
weight = pm.Normal('weight', mu=mu_weight, sd=sd_weight, observed=d2['weight'])

```

---

<div class="post-metadata">

**Author:** ![Paras\_Chopra](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/paras_chopra/32/1385_2.png) [@Paras\_Chopra](https://discourse.pymc.io/u/Paras_Chopra)\
**Post date:** [January 7, 2019, 3:51pm UTC](https://discourse.pymc.io/t/linear-regression-difference-between-these-two-models/2487/3 "2019-01-07T15:51:59Z")

</div>

Thanks. A quick follow up question on your comment:

> Moreover, `weight` will be a tensor with value set to `d2['weight']` .

Since, mu\_height is determined by:

mu\_height = alpha + beta \* (d2[‘weight’]-np.mean(d2[‘weight’]))

I’m assuming mu\_height will also be a tensor (not tensor) during sampling? Is my understanding correct?

In that case, what does mu=mu\_height imply in:

height = pm.Normal(‘height’, mu=mu\_height, sd=sigma\_height, observed=d2[‘height’])

My assumption was that mu took a scalar value. How is mu\_height tensor treated? (Also sigma\_height isn’t a tensor and I’m not sure why that won’t cause a problem)

---

<div class="post-metadata">

**Author:** ![junpenglao](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/junpenglao/32/8_2.png) [@junpenglao](https://discourse.pymc.io/u/junpenglao)\
**Post date:** [January 8, 2019, 2:05am UTC](https://discourse.pymc.io/t/linear-regression-difference-between-these-two-models/2487/4 "2019-01-08T02:05:23Z")

</div>

> [@Paras\_Chopra](#):
>
> I’m assuming mu\_height will also be a tensor (not tensor) during sampling? Is my understanding correct?

Yes

> [@Paras\_Chopra](#):
>
> My assumption was that mu took a scalar value. How is mu\_height tensor treated?

It will be a vector tensor with the shape same as the observed. Its value is propagated from other free parameters, so it is essentially invisible to the sampler (as it is not a free parameter).

> [@Paras\_Chopra](#):
>
> (Also sigma\_height isn’t a tensor and I’m not sure why that won’t cause a problem)

It will be convert into a tensor constant during run time

---

<div class="post-metadata">

**Author:** ![Paras\_Chopra](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/paras_chopra/32/1385_2.png) [@Paras\_Chopra](https://discourse.pymc.io/u/Paras_Chopra)\
**Post date:** [January 8, 2019, 6:13am UTC](https://discourse.pymc.io/t/linear-regression-difference-between-these-two-models/2487/5 "2019-01-08T06:13:28Z")

</div>

Thanks.

> [@junpenglao](#):
>
> > (Also sigma\_height isn’t a tensor and I’m not sure why that won’t cause a problem)
> 
> It will be convert into a tensor constant during run time

So this means if mu\_height is of shape (100) containing 100 different data points, sigma\_height will also be recasted as shape of (100) with the values _replicated_ 100 times?

---

<div class="post-metadata">

**Author:** ![junpenglao](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/junpenglao/32/8_2.png) [@junpenglao](https://discourse.pymc.io/u/junpenglao)\
**Post date:** [January 8, 2019, 7:50am UTC](https://discourse.pymc.io/t/linear-regression-difference-between-these-two-models/2487/6 "2019-01-08T07:50:57Z")

</div>

It will be broadcasted during the computation of logp. But will not be a shape (100) tensor the same sense as mu\_height.