# Multiple chains in Variational inference

**URL:** <https://discourse.pymc.io/t/multiple-chains-in-variational-inference/3805>\
**Category:** Questions\
**Created:** [August 30, 2019, 6:19pm UTC](https://discourse.pymc.io/t/multiple-chains-in-variational-inference/3805 "2019-08-30T18:19:32Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![indian312](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/indian312/32/2117_2.png) [@indian312](https://discourse.pymc.io/u/indian312)\
**Post date:** [August 30, 2019, 6:19pm UTC](https://discourse.pymc.io/t/multiple-chains-in-variational-inference/3805/1 "2019-08-30T18:19:32Z")

</div>

1. I would like to use multiple chains for ADVI. So far, i could not find a single documentation on **how to sample multiple chains in ADVI?**

2. If multiple chains are not possible, i would like to append each run of pm.variational.approximations.sample\_approx (…) such that i get a final trace of type

> “\<MultiTrace: ‘n’ chains, ‘x’ iterations, ‘m’ variables\>”  
> from appending ‘n’ samples of  
> “\<MultiTrace: ‘1’ chains, ‘x’ iterations, ‘m’ variables\>”

**How to append ?**

---

<div class="post-metadata">

**Author:** ![indian312](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/indian312/32/2117_2.png) [@indian312](https://discourse.pymc.io/u/indian312)\
**Post date:** [August 30, 2019, 7:35pm UTC](https://discourse.pymc.io/t/multiple-chains-in-variational-inference/3805/2 "2019-08-30T19:35:47Z")

</div>

For the second question ‘2.’ I found a solution,

> Multi\_chain\_trace =   
> for i in range(required\_chain\_number) : #Repeat over _required\_chain\_number_#Loop begin  
> approx\_trace = pm.variational.inference.fit(n = _no\_of\_draws\_required_, method = ‘advi’)  
> single\_chain\_trace = approx\_trace.sample(_no\_of\_draws\_required_)  
> Multi\_chain\_trace.append(single\_chain\_trace)  
> Multi\_chain\_trace[i].\_straces[i] = Multi\_chain\_trace[i].\_straces.pop(0)
> 
> #End of Loop
> 
> Multi\_chain\_trace = pm.backends.base.merge\_traces(Multi\_chain\_trace)
> 
> pm.traceplot(Multi\_chain\_trace)

One can append single chains this way.

---

<div class="post-metadata">

**Author:** ![chartl](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/chartl/32/1515_2.png) [@chartl](https://discourse.pymc.io/u/chartl)\
**Post date:** [September 1, 2019, 5:58pm UTC](https://discourse.pymc.io/t/multiple-chains-in-variational-inference/3805/3 "2019-09-01T17:58:34Z")

</div>

Multiple chains are typically used to check MCMC convergence criteria. These criteria do not make sense for ADVI, as an approximating Gaussian is fit in closed form. The sampling you get from ADVI does not involve MCMC at all – and so there’s no such thing as a “chain” for ADVI – which is why there is no documentation on the subject.