# Memory spike at the end of the MCMC sampling

**URL:** <https://discourse.pymc.io/t/memory-spike-at-the-end-of-the-mcmc-sampling/5669>\
**Category:** Questions\
**Created:** [August 21, 2020, 10:39am UTC](https://discourse.pymc.io/t/memory-spike-at-the-end-of-the-mcmc-sampling/5669 "2020-08-21T10:39:22Z")\
**Posts on this page:** 1\
**Showing post:** 2

<div class="post-metadata">

**Author:** ![OriolAbril](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/oriolabril/32/2497_2.png) [@OriolAbril](https://discourse.pymc.io/u/OriolAbril)\
**Post date:** [August 21, 2020, 1:27pm UTC](https://discourse.pymc.io/t/memory-spike-at-the-end-of-the-mcmc-sampling/5669/2 "2020-08-21T13:27:26Z")

</div>

The issue is due to storing pointwise log likelihood values, a step which is done at the end of sampling when calculating ess and rhat. The default is to store such data because it is required for loo/waic calculation and further model comparison.

Taking a look at the answers to

> [@DE-Metropolis with blackbox likelihood does not terminate](https://discourse.pymc.io/t/de-metropolis-with-blackbox-likelihood-does-not-terminate/5360):
>
> I am trying to run DE-MCMC with a Blackbox likelihood function. I have 3 parameters that go into the blackbox likelihood function. The program starts running and gives me the following output. It doesn’t really terminate even when I leave it running for a long time. Any idea why this is taking too long or maybe not terminating at all? Population sampling (4 chains) DEMetropolis: [a1, a2, a3] Attempting to parallelize chains to all cores. You can turn this off with `pm.sample(cores=1)`. ... Po…

should give more details and guidance on avoiding the issue.

I am also interested in these use cases when due to large number of observations pointwise log likelihood or posterior predictive do not fit in memory. We are working on integrating Dask with ArviZ (see work started on [https://github.com/arviz-devs/arviz/pull/1229](https://github.com/arviz-devs/arviz/pull/1229)) to eventually allow ppc checks and loo/waic calculation for these models.

---

_[View the full topic](https://discourse.pymc.io/t/memory-spike-at-the-end-of-the-mcmc-sampling/5669)._