# Can different samples be merged?

**URL:** https://discourse.pymc.io/t/can-different-samples-be-merged/3184
**Category:** Questions
**Created:** [May 1, 2019, 5:11pm UTC](https://discourse.pymc.io/t/can-different-samples-be-merged/3184 "2019-05-01T17:11:00Z")
**Posts on this page:** 1
**Showing post:** 5

<div class="post-metadata">

### Author: ![chartl](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/chartl/32/1515_2.png) [@chartl](https://discourse.pymc.io/u/chartl)
#### Post date: [May 2, 2019, 6:05pm UTC](https://discourse.pymc.io/t/can-different-samples-be-merged/3184/5 "2019-05-02T18:05:12Z")

</div>

In my comment, “chain” refers to a single sequence of `n` NUTS draws from the posterior, starting at some initial (random) point. Re-setting to a different starting point and generating a new sequence therefore generates a new chain.

By default, these are done in parallel in PyMC3, with one core of a CPU devoted to sampling from a single chain.

This can be expanded to multiple CPUs by providing each CPU with a copy of the entire dataset.

Splitting the dataset up across multiple CPUs (“batching”) is another matter entirely, and is an active area of research generally. See here for some ideas:

> [@Using MCMC based model in closed loop](https://discourse.pymc.io/t/using-mcmc-based-model-in-closed-loop/3145/8):
>
> There are a couple of cases that work trivially out of the box. Variational approximations work just fine, since at each presentation of a new dataset we have \pi\_{k+1}(\theta) = \mathrm{argmin}\_\mathcal{q} \mathrm{KL}[q(\theta) || P(D\_{k+1}|\theta)\pi\_k(\theta)] so the information propagates. The drawback is that none of the \pi\_i will converge to the true posterior. Resampling methods also work out of the box, since the weights update in the same cumulative way: w\_i^{(k+1)} = w\_i^{(k)}P[D…

---

_[View the full topic](https://discourse.pymc.io/t/can-different-samples-be-merged/3184)._