# Bayesian linear regression with SVGD

**URL:** <https://discourse.pymc.io/t/bayesian-linear-regression-with-svgd/6608>\
**Category:** Questions\
**Created:** [January 17, 2021, 4:11am UTC](https://discourse.pymc.io/t/bayesian-linear-regression-with-svgd/6608 "2021-01-17T04:11:00Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Eric\_Chan](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/eric_chan/32/3573_2.png) [@Eric\_Chan](https://discourse.pymc.io/u/Eric_Chan)\
**Post date:** [January 17, 2021, 4:11am UTC](https://discourse.pymc.io/t/bayesian-linear-regression-with-svgd/6608/1 "2021-01-17T04:11:00Z")

</div>

Hi, I am using stein variational gradient descent to solve a simple Bayesian linear regression problem, but the result seems to have some problems for the variance parameter. My code does not use pymc3, and can anyone provide a sample code in pymc3 for this problem so I can verify whether this is my own mistake or a limitation of the SVGD algorithm.

---

<div class="post-metadata">

**Author:** ![michaelosthege](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/michaelosthege/32/371_2.png) [@michaelosthege](https://discourse.pymc.io/u/michaelosthege)\
**Post date:** [January 17, 2021, 6:12pm UTC](https://discourse.pymc.io/t/bayesian-linear-regression-with-svgd/6608/2 "2021-01-17T18:12:47Z")

</div>

Hi, I’m not familiar with SVGD myself, but have you checked out the examples or tutorials from our gallery?

→ [tutorials\_notebooks — PyMC3 3.10.0 documentation](https://docs.pymc.io/nb_tutorials/index.html) section Tutorials/Basics/Variational API Quickstart  
→ [examples\_notebooks — PyMC3 3.10.0 documentation](https://docs.pymc.io/nb_examples/index.html) section Examples/Variational Inference

---

<div class="post-metadata">

**Author:** ![Eric\_Chan](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/eric_chan/32/3573_2.png) [@Eric\_Chan](https://discourse.pymc.io/u/Eric_Chan)\
**Post date:** [January 18, 2021, 3:50am UTC](https://discourse.pymc.io/t/bayesian-linear-regression-with-svgd/6608/3 "2021-01-18T03:50:14Z")

</div>

Hi michael, I have checked these exampes and I got the following ADVI example, but it cannot work

```python
import matplotlib.pyplot as plt
import numpy as np
import pymc3 as pm
from pymc3 import *
import pandas as pd
import theano

size = 200
true_intercept = 1
true_slope = 2

x = np.linspace(0, 1, size)

# y = a + b*x
true_regression_line = true_intercept + true_slope * x
# add noise
y = true_regression_line + np.random.normal(scale=0.5, size=size)

data = dict(x=x, y=y)

# show the data
fig = plt.figure(figsize=(7, 7))
ax = fig.add_subplot(111, xlabel="x", ylabel="y", title="Generated data and underlying model")
ax.plot(x, y, "x", label="sampled data")
ax.plot(x, true_regression_line, label="true regression line", lw=2.0)
plt.legend(loc=0)
plt.show()

with Model() as model: # model specifications in PyMC3 are wrapped in a with-statement
    # Define priors
    sigma = HalfNormal("sigma", sd=1)
    intercept = Normal("Intercept", 0, sd=1)
    x_coeff = Normal("x", 0, sd=1)

    # Define likelihood
    likelihood = Normal("y", mu=intercept + x_coeff * x, sd=sigma, observed=y)

    # Inference!
    trace = sample(3000, cores=2) # draw 3000 posterior samples using NUTS sampling

    # traceplot(trace)

    # mean_field = pm.fit(method="advi")
    advi_fit = pm.fit(method=pm.ADVI(), n=30000)

    pm.plot_posterior(advi_fit.sample(1000), color="LightSeaGreen")

```

The nuts sampling is OK, but the ADVI will raise errors.

---

<div class="post-metadata">

**Author:** ![Dominik](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/dominik/32/918_2.png) [@Dominik](https://discourse.pymc.io/u/Dominik)\
**Post date:** [January 19, 2021, 4:12pm UTC](https://discourse.pymc.io/t/bayesian-linear-regression-with-svgd/6608/4 "2021-01-19T16:12:28Z")

</div>

What error is raised exactly? Your code works fine in my jupyter notebook running PyMC 3.10.0. The resulting plot:

 ![grafik](https://canada1.discourse-cdn.com/flex036/uploads/pymc3/original/2X/2/2d256dbbe339244aa53a17f34de900a2f8e98797.png)

**Edit:** For some reason the image is shown with black background hiding the labels. From left to right, these are the posteriors for `Intercept`, `x`, and `sigma`.

---

<div class="post-metadata">

**Author:** ![Eric\_Chan](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/eric_chan/32/3573_2.png) [@Eric\_Chan](https://discourse.pymc.io/u/Eric_Chan)\
**Post date:** [January 21, 2021, 4:24am UTC](https://discourse.pymc.io/t/bayesian-linear-regression-with-svgd/6608/5 "2021-01-21T04:24:05Z")

</div>

Hi Dominik, thanks for your reply. I find it may be the problem of theano. I reinstall pymc3 and the code now works normally.