# Pause and resume sampling while changing the model

**URL:** <https://discourse.pymc.io/t/pause-and-resume-sampling-while-changing-the-model/11557>\
**Category:** v5\
**Created:** [March 8, 2023, 11:54am UTC](https://discourse.pymc.io/t/pause-and-resume-sampling-while-changing-the-model/11557 "2023-03-08T11:54:03Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![ibi](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/ibi/32/6197_2.png) [@ibi](https://discourse.pymc.io/u/ibi)\
**Post date:** [March 8, 2023, 11:54am UTC](https://discourse.pymc.io/t/pause-and-resume-sampling-while-changing-the-model/11557/1 "2023-03-08T11:54:03Z")

</div>

I am trying to use PyMC for a Bayesian Inversion Problem. I would need to start with a prior, get some samples using MCMC, then change my forward function and resume sampling, i.e., I do not want to start again with the original prior. Is there a way to do it?

For the moment, it might be enough to use the standard Metropolis-Hasting sampler. My intuition is that, since we are working with Markov Chains, it should be enough to recover the last state of the previous trace and resume sampling from that. However, I noticed that using `pm.sample` with the argument `trace=previous_trace` does not work anymore, while in PyMC3 it was.

Do I achieve what I am willing to if I initialize a sampler once at the very beginning, say `sampler = pm.Metropolis()` or whatever sampler, and then use `pm.sample(step=sampler)`? I did not understand if this is indeed a solution or not: it seems to produce some meaningful results, but still I did not understand completely how it works.

I also tried to follow the suggestions in [this post](https://discourse.pymc.io/t/resuming-sampling-from-a-previous-trace/11245) and the linked ones, but I did not figure it out. Does anybody know if there exists a solution to that? A hard-coded one would be enough for the moment.

---

<div class="post-metadata">

**Author:** ![jessegrabowski](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/jessegrabowski/32/5010_2.png) [@jessegrabowski](https://discourse.pymc.io/u/jessegrabowski)\
**Post date:** [March 8, 2023, 12:22pm UTC](https://discourse.pymc.io/t/pause-and-resume-sampling-while-changing-the-model/11557/2 "2023-03-08T12:22:29Z")

</div>

Would something like converting the previous posterior to a prior for your new model, as in [this example](https://www.pymc.io/projects/examples/en/latest/howto/updating_priors.html), work? It seems like you want to iterate on the model without throwing away good information from the previous runs, which this scheme would accomplish. I’m not an expert on MCMC theory, but I think changing the model then resuming sampling from a past chain would bias the result (my intuition: the first half of the chain would be from the original model, while the other half would be from the new model, and thus the whole chain would represent neither).

If all you care about is the starting points, you can provide a dictionary of initial values to `pm.sample` via the `initvals` argument. As long as all the variable names stayed the same between models, it would be easy enough to write a function that takes in an `idata`, pops off the last samples for each variable, and returns a dictionary.

---

<div class="post-metadata">

**Author:** ![ibi](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/ibi/32/6197_2.png) [@ibi](https://discourse.pymc.io/u/ibi)\
**Post date:** [March 8, 2023, 2:29pm UTC](https://discourse.pymc.io/t/pause-and-resume-sampling-while-changing-the-model/11557/3 "2023-03-08T14:29:49Z")

</div>

Hi, thank you for your answer. The example you pointed me to is not appropriate for my use case because I would like to avoid drawing samples from the posterior and using them as prior for the new model. Moreover, I have good reasons to change the model, and for my application, it will not bias the results.

I would like to do something like:

```auto
with model_1:
     trace_1 = pm.sample(1000)

#
#some modifications to model_1
#

with model_1:
    trace_2 = pm.sample(1000, tune=0)

```

and then gather `trace_1` and `trace_2` in a single `trace`, in a way that if I didn’t do anything to the model, the code above would be equivalent to

```auto
with model_1:
     trace = pm.sample(2000)

```

Thanks for your help!

---

<div class="post-metadata">

**Author:** ![wmbelk](https://avatars.discourse-cdn.com/v4/letter/w/ecccb3/32.png) [@wmbelk](https://discourse.pymc.io/u/wmbelk)\
**Post date:** [March 10, 2023, 4:44am UTC](https://discourse.pymc.io/t/pause-and-resume-sampling-while-changing-the-model/11557/4 "2023-03-10T04:44:33Z")

</div>

If you want the results joined, could you just use arviz to merge the inference data outputs?

---

<div class="post-metadata">

**Author:** ![ibi](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/ibi/32/6197_2.png) [@ibi](https://discourse.pymc.io/u/ibi)\
**Post date:** [March 10, 2023, 8:47am UTC](https://discourse.pymc.io/t/pause-and-resume-sampling-while-changing-the-model/11557/5 "2023-03-10T08:47:42Z")

</div>

I don’t want the results to be joined, I would like to stop and resume sampling without starting again from my initial uninformative prior.

---

<div class="post-metadata">

**Author:** ![cluhmann](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/cluhmann/32/3083_2.png) [@cluhmann](https://discourse.pymc.io/u/cluhmann)\
**Post date:** [March 10, 2023, 2:40pm UTC](https://discourse.pymc.io/t/pause-and-resume-sampling-while-changing-the-model/11557/6 "2023-03-10T14:40:21Z")

</div>

See related discussion [here](https://discourse.pymc.io/t/resuming-sampling-from-a-previous-trace/11245/5), [here](https://discourse.pymc.io/t/saving-intermediate-results-using-mcmc-in-pymc4/9938/4), and [here](https://discourse.pymc.io/t/saving-intermediate-traces/3313/5). I recommend looking at [GitHub - michaelosthege/mcbackend: A backend for storing MCMC draws.](https://github.com/michaelosthege/mcbackend) written by @michaelosthege .