# Intro Bayesian Regression using HMC & ADVI

**URL:** <https://discourse.pymc.io/t/intro-bayesian-regression-using-hmc-advi/2165>\
**Category:** Sharing\
**Created:** [November 7, 2018, 11:57pm UTC](https://discourse.pymc.io/t/intro-bayesian-regression-using-hmc-advi/2165 "2018-11-07T23:57:49Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![AlexIoannides](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/alexioannides/32/1524_2.png) [@AlexIoannides](https://discourse.pymc.io/u/AlexIoannides)\
**Post date:** [November 7, 2018, 11:57pm UTC](https://discourse.pymc.io/t/intro-bayesian-regression-using-hmc-advi/2165/1 "2018-11-07T23:57:49Z")

</div>

Hello,

I wrote-up my notes/notebook on practical use of ADVI, into a blog post that might be useful (primarily for ‘beginners’):

> **[Bayesian Regression in PYMC3 using MCMC & Variational Inference](https://alexioannides.com/2018/11/07/bayesian-regression-in-pymc3-using-mcmc-variational-inference/)**
>
> Conducting a Bayesian data analysis - e.g. estimating a Bayesian linear regression model - will usually require some form of Probabilistic Programming Language (PPL), unless analytical approaches (e.g. based on conjugate prior models), are...

The GitHub repo with the notebook, is here:

> **[AlexIoannides/pymc-advi-hmc-demo](https://github.com/AlexIoannides/pymc-advi-hmc-demo)**
>
> Demonstrating HMC and ADVI algorithms for Bayesian data analysis using PYMC3. - AlexIoannides/pymc-advi-hmc-demo

Alex

---

<div class="post-metadata">

**Author:** ![ferrine](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/ferrine/32/10_2.png) [@ferrine](https://discourse.pymc.io/u/ferrine)\
**Post date:** [November 20, 2018, 8:29am UTC](https://discourse.pymc.io/t/intro-bayesian-regression-using-hmc-advi/2165/2 "2018-11-20T08:29:46Z")

</div>

Hi, nice read. Some nitpicks:

> that do **not** rely heavily on computationally expensive random sampling

This needs to be rephrased. Is is more about exact full data gradient and stochastic one.

Simulations using full rank advi might be informative as well (posterior resembles gaussian with correlations)

---

<div class="post-metadata">

**Author:** ![AlexIoannides](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/alexioannides/32/1524_2.png) [@AlexIoannides](https://discourse.pymc.io/u/AlexIoannides)\
**Post date:** [November 20, 2018, 10:02am UTC](https://discourse.pymc.io/t/intro-bayesian-regression-using-hmc-advi/2165/3 "2018-11-20T10:02:58Z")

</div>

Thanks for taking the time to read it - greatly appreciated.

I don’t understand - you’ll have to be more explicit.

The processes of ‘fitting’ a model is more computationally expensive for MCMC (random sampling), when compared to ADVI (zero or full-rank). I realise that for ADVI one has still to sample from the ‘fit’ before anything can be inferred, however - is this what you’re getting at?

---

<div class="post-metadata">

**Author:** ![ferrine](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/ferrine/32/10_2.png) [@ferrine](https://discourse.pymc.io/u/ferrine)\
**Post date:** [December 16, 2018, 6:18pm UTC](https://discourse.pymc.io/t/intro-bayesian-regression-using-hmc-advi/2165/4 "2018-12-16T18:18:45Z")

</div>

Sorry for the late answer (notification problems from my side). The posterior distribution seemed to be correlated and fitting variatioal distribution, eg FullRankADVI might be beneficial. It will show the reader that we can have both: efficient fitting + good approximation.

---

<div class="post-metadata">

**Author:** ![AlexIoannides](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/alexioannides/32/1524_2.png) [@AlexIoannides](https://discourse.pymc.io/u/AlexIoannides)\
**Post date:** [January 18, 2019, 10:48am UTC](https://discourse.pymc.io/t/intro-bayesian-regression-using-hmc-advi/2165/5 "2019-01-18T10:48:38Z")

</div>

Fair point - alluded to at the end, but not explicit.