# Sampling does not start or very very slow while attempting a Mixture Model tutorial

**URL:** <https://discourse.pymc.io/t/sampling-does-not-start-or-very-very-slow-while-attempting-a-mixture-model-tutorial/1882>\
**Category:** Questions\
**Created:** [September 9, 2018, 4:01am UTC](https://discourse.pymc.io/t/sampling-does-not-start-or-very-very-slow-while-attempting-a-mixture-model-tutorial/1882 "2018-09-09T04:01:40Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![ikibkilam1](https://avatars.discourse-cdn.com/v4/letter/i/ea5d25/32.png) [@ikibkilam1](https://discourse.pymc.io/u/ikibkilam1)\
**Post date:** [September 9, 2018, 4:01am UTC](https://discourse.pymc.io/t/sampling-does-not-start-or-very-very-slow-while-attempting-a-mixture-model-tutorial/1882/1 "2018-09-09T04:01:40Z")

</div>

I am new to PYMC3. While learning, I did a Mixture Model tutorial, outlined in the book “Probabilistic Programming & Bayesian Methods for Hackers”. I believe I am doing exactly what was outlined in the tutorial, however my sampling does not start or is very slow. I tried various permutations, as below, without success. Note, the data size is 300. Could someone tell me what I am doing wrong?

```
with pm.Model() as model:
p1 = pm.Uniform('p1', 0, 1)
p2 = 1 - p1
p = T.stack([p1, p2])
assign = pm.Categorical('assign', p, shape = data.values.shape[0], testval = np.random.randint
                       (0, 2, data.values.shape[0]))
sdevs = pm.Uniform('sdevs', 0, 100, shape = 2)
means = pm.Normal('means', mu = np.array([120, 190]), sd = np.array([10, 10]), shape = 2)
means_ = pm.Deterministic("means_", means[assign])
sdevs_ = pm.Deterministic("sdevs_", sdevs[assign])
obs = pm.Normal('obs', mu = means_, sd = sdevs_, observed = data.values)`

```

First, I let PyMC decide which routines to use and saw no results in 5 hours:

```
with model:
trace = pm.sample(25000)
Multiprocess sampling (4 chains in 4 jobs)
CompoundStep
> NUTS: [means, sdevs, p1]
> BinaryGibbsMetropolis: [assign]
> Sampling 4 chains: 0%| | 24/102000 [5:03:59&lt;40738:03:24, 1438.15s/draws

```

I also tried to be more prescriptive and attempted the following, but the sampler is very slow:

```
with model:
start = find_MAP(model = model)
step1 = pm.Metropolis(vars = [p, sdevs, means])
step2 = pm.ElemwiseCategorical(vars = [assign])
trace = pm.sample(25000, start = start, step = [step1, step2])
logp = -4.7305e+05, ||grad|| = 299.51: 100%|██ █████████████████████████████████████████| 39/39 [01:37&lt;00:00, 2.51s/it]
C:\Users\bikim\AppData\Local\conda\conda\envs\pymc3p36\lib\site-packages\ipykernel_launcher.py:6: 
DeprecationWarning: ElemwiseCategorical is deprecated, switch to CategoricalGibbsMetropolis.

Multiprocess sampling (4 chains in 4 jobs)
CompoundStep
> CompoundStep
> > Metropolis: [means]
> > Metropolis: [sdevs]
> > Metropolis: [p1]
> > ElemwiseCategorical: [assign]
> > Sampling 4 chains: 1%|▎ | 552/102000 [30:35&lt;105:28:23, 3.74s/draws]

```

This should be an easy problem. I appreciate any insight into what I am doing wrong.

---

<div class="post-metadata">

**Author:** ![junpenglao](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/junpenglao/32/8_2.png) [@junpenglao](https://discourse.pymc.io/u/junpenglao)\
**Post date:** [September 9, 2018, 11:01am UTC](https://discourse.pymc.io/t/sampling-does-not-start-or-very-very-slow-while-attempting-a-mixture-model-tutorial/1882/2 "2018-09-09T11:01:27Z")

</div>

Mixture model parameterized with explicit latent label is _really_ difficult to sampled from, the recommended solution is to rewrite it into a marginalized mixture model - you can do a search in this discourse there are quite a few related discussions.

---

<div class="post-metadata">

**Author:** ![\_eigenfoo](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/_eigenfoo/32/3059_2.png) [@\_eigenfoo](https://discourse.pymc.io/u/_eigenfoo)\
**Post date:** [September 10, 2018, 2:17am UTC](https://discourse.pymc.io/t/sampling-does-not-start-or-very-very-slow-while-attempting-a-mixture-model-tutorial/1882/3 "2018-09-10T02:17:11Z")

</div>

Following up on Junpeng’s answer, there’s a good example of marginalized mixture models in the docs: [https://docs.pymc.io/notebooks/marginalized\_gaussian\_mixture\_model.html](https://docs.pymc.io/notebooks/marginalized_gaussian_mixture_model.html)

---

<div class="post-metadata">

**Author:** ![ikibkilam1](https://avatars.discourse-cdn.com/v4/letter/i/ea5d25/32.png) [@ikibkilam1](https://discourse.pymc.io/u/ikibkilam1)\
**Post date:** [September 19, 2018, 4:29pm UTC](https://discourse.pymc.io/t/sampling-does-not-start-or-very-very-slow-while-attempting-a-mixture-model-tutorial/1882/4 "2018-09-19T16:29:32Z")

</div>

Thank you both (\_eigenfoo and junpenglao). It works like a charm now.

---

<div class="post-metadata">

**Author:** ![\_eigenfoo](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/_eigenfoo/32/3059_2.png) [@\_eigenfoo](https://discourse.pymc.io/u/_eigenfoo)\
**Post date:** [September 19, 2018, 10:15pm UTC](https://discourse.pymc.io/t/sampling-does-not-start-or-very-very-slow-while-attempting-a-mixture-model-tutorial/1882/5 "2018-09-19T22:15:25Z")

</div>

Glad to hear!