# HMC-based kernel for SMC

**URL:** <https://discourse.pymc.io/t/hmc-based-kernel-for-smc/6250>\
**Category:** Questions\
**Created:** [November 12, 2020, 6:53pm UTC](https://discourse.pymc.io/t/hmc-based-kernel-for-smc/6250 "2020-11-12T18:53:57Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![hessammehr](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/hessammehr/32/3283_2.png) [@hessammehr](https://discourse.pymc.io/u/hessammehr)\
**Post date:** [November 12, 2020, 6:53pm UTC](https://discourse.pymc.io/t/hmc-based-kernel-for-smc/6250/1 "2020-11-12T18:53:57Z")

</div>

I was wondering if anyone familiar with the PyMC3 codebase had opinions on integrating a Hamiltonian kernel into PyMC3’s Sequential Monte Carlo implementation. How much work do you think it would be?

**Motivation:** I haven’t had much luck using NUTS to sample heavily multimodal, high (\> 1000)-dimensional posteriors. Luckily, the prior distribution is fairly smooth (possibly quite common in the real world), so I would happily use SMC if it weren’t for the inefficient Metropolis step. With a continuously-tuned HMC kernel it might be possible to get the best of both worlds.

Depending on your feedback I might try and have a go at implementing this. So far I’ve found [this paper](https://arxiv.org/pdf/1808.07730.pdf), which seems to support the hypothesis. Any pointers on how best to approach the problem would be greatly appreciated.

---

<div class="post-metadata">

**Author:** ![junpenglao](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/junpenglao/32/8_2.png) [@junpenglao](https://discourse.pymc.io/u/junpenglao)\
**Post date:** [November 13, 2020, 6:59am UTC](https://discourse.pymc.io/t/hmc-based-kernel-for-smc/6250/2 "2020-11-13T06:59:39Z")

</div>

Before you go ahead and implement it, you might want to check whether SMC with HMC internal kernel actually work in your model. I wrote the SMC in TFP where you can choose different internal kernel (including HMC), which you could give it a try: [https://github.com/tensorflow/probability/blob/master/tensorflow\_probability/python/experimental/mcmc/examples/smc\_demo.ipynb](https://github.com/tensorflow/probability/blob/master/tensorflow_probability/python/experimental/mcmc/examples/smc_demo.ipynb)

---

<div class="post-metadata">

**Author:** ![junpenglao](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/junpenglao/32/8_2.png) [@junpenglao](https://discourse.pymc.io/u/junpenglao)\
**Post date:** [November 13, 2020, 2:45pm UTC](https://discourse.pymc.io/t/hmc-based-kernel-for-smc/6250/3 "2020-11-13T14:45:09Z")

</div>

Also see [this example](https://gist.github.com/junpenglao/637647b975e3616f5bd362a07859d1d8) where I directly replicating what pymc3 is currently doing with the independent MH inner kernel for SMC.

---

<div class="post-metadata">

**Author:** ![hessammehr](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/hessammehr/32/3283_2.png) [@hessammehr](https://discourse.pymc.io/u/hessammehr)\
**Post date:** [November 16, 2020, 10:02am UTC](https://discourse.pymc.io/t/hmc-based-kernel-for-smc/6250/4 "2020-11-16T10:02:28Z")

</div>

Thanks very much for the links. At the moment I am trying to work out which parts of your notebooks are TF-specific and which can be reused with PyMC3. I might have a look at the PyMC3 SMS code next and see where the kernel comes into play.

Thanks again @junpenglao for pointing me in the right direction.