# Kernel is now dying with on a model that ran before

**URL:** <https://discourse.pymc.io/t/kernel-is-now-dying-with-on-a-model-that-ran-before/9525>\
**Category:** v5\
**Tags:** modeling\
**Created:** [May 30, 2022, 10:38am UTC](https://discourse.pymc.io/t/kernel-is-now-dying-with-on-a-model-that-ran-before/9525 "2022-05-30T10:38:12Z")\
**Posts on this page:** 15\
**Page:** 1

<div class="post-metadata">

**Author:** ![jordan.howell2](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/jordan.howell2/32/1742_2.png) [@jordan.howell2](https://discourse.pymc.io/u/jordan.howell2)\
**Post date:** [May 30, 2022, 10:38am UTC](https://discourse.pymc.io/t/kernel-is-now-dying-with-on-a-model-that-ran-before/9525/1 "2022-05-30T10:38:12Z")

</div>

Hello. I’ve been developing a model that has successfully sampled multiple times in a google cloud platform environment. As of four days ago, it quit sampling successfully to be replaced with a message that the “kernel expectantly died.”

I thought this may be because I changed my observed distribution from Normal to Poisson but it did the same thing when switching back to normal.

I’m not getting any new warnings. The only warning I’ve gotten was the  
`/opt/conda/lib/python3.7/site-packages/pymc/aesaraf.py:1010: UserWarning: The parameter 'updates' of aesara.function() expects an OrderedDict, got <class 'dict'>. Using a standard dictionary here results in non-deterministic behavior. You should use an OrderedDict if you are using Python 2.7 (collections.OrderedDict for older python), or use a list of (shared, update) pairs. Do not just convert your dictionary to this type before the call as the conversion will still be non-deterministic.`

I took the code in the notebook and ran it as a script through the console and had this new message pop up:

```auto
  **kwargs,
Segmentation fault

```

I’m not sure if that has something to do with it. Has anyone ran into this problem before?

I’m running in a Linux environment.

PyMC Version: 4.0.0b6  
Aesara Version: 2.5.1  
Arvize Verions: 0.12.1

---

<div class="post-metadata">

**Author:** ![cluhmann](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/cluhmann/32/3083_2.png) [@cluhmann](https://discourse.pymc.io/u/cluhmann)\
**Post date:** [June 1, 2022, 3:17am UTC](https://discourse.pymc.io/t/kernel-is-now-dying-with-on-a-model-that-ran-before/9525/2 "2022-06-01T03:17:22Z")

</div>

How did you install PyMC? Your version of aesara doesn’t match [the requirements](https://github.com/pymc-devs/pymc/blob/main/requirements.txt) for PyMC v4, so something may have gotten messed up when you installed PyMC and/or created your environment.

---

<div class="post-metadata">

**Author:** ![jordan.howell2](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/jordan.howell2/32/1742_2.png) [@jordan.howell2](https://discourse.pymc.io/u/jordan.howell2)\
**Post date:** [June 2, 2022, 1:15am UTC](https://discourse.pymc.io/t/kernel-is-now-dying-with-on-a-model-that-ran-before/9525/3 "2022-06-02T01:15:02Z")

</div>

I installed by typing

`pip install pymc --pre`

I’ll update tomorrow morning. Thank you.

---

<div class="post-metadata">

**Author:** ![jordan.howell2](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/jordan.howell2/32/1742_2.png) [@jordan.howell2](https://discourse.pymc.io/u/jordan.howell2)\
**Post date:** [June 2, 2022, 10:24am UTC](https://discourse.pymc.io/t/kernel-is-now-dying-with-on-a-model-that-ran-before/9525/4 "2022-06-02T10:24:33Z")

</div>

Just tried to upgrade aesara and got the following error:

`ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. pymc 4.0.0b6 requires aesara==2.5.1, but you have aesara 2.6.2 which is incompatible.`

---

<div class="post-metadata">

**Author:** ![DanhPhan](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/danhphan/32/4799_2.png) [@DanhPhan](https://discourse.pymc.io/u/DanhPhan)\
**Post date:** [June 2, 2022, 11:11am UTC](https://discourse.pymc.io/t/kernel-is-now-dying-with-on-a-model-that-ran-before/9525/5 "2022-06-02T11:11:32Z")

</div>

Hi, you can try install the latest pymc version (along with latest aesara and aeppl) from github repos by using this command:

`pip install git+https://github.com/pymc-devs/pymc.git`

---

<div class="post-metadata">

**Author:** ![cluhmann](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/cluhmann/32/3083_2.png) [@cluhmann](https://discourse.pymc.io/u/cluhmann)\
**Post date:** [June 2, 2022, 1:20pm UTC](https://discourse.pymc.io/t/kernel-is-now-dying-with-on-a-model-that-ran-before/9525/6 "2022-06-02T13:20:59Z")

</div>

I would also follow the instructions found [here](https://www.pymc.io/projects/docs/en/latest/installation.html) from scratch rather than try to fix a broken instillation by upgrading.

---

<div class="post-metadata">

**Author:** ![jordan.howell2](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/jordan.howell2/32/1742_2.png) [@jordan.howell2](https://discourse.pymc.io/u/jordan.howell2)\
**Post date:** [June 2, 2022, 1:43pm UTC](https://discourse.pymc.io/t/kernel-is-now-dying-with-on-a-model-that-ran-before/9525/7 "2022-06-02T13:43:01Z")

</div>

Thank you. I did follow those instructions first and that’s what gave me the older version of aesara.

---

<div class="post-metadata">

**Author:** ![jordan.howell2](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/jordan.howell2/32/1742_2.png) [@jordan.howell2](https://discourse.pymc.io/u/jordan.howell2)\
**Post date:** [June 3, 2022, 2:20pm UTC](https://discourse.pymc.io/t/kernel-is-now-dying-with-on-a-model-that-ran-before/9525/8 "2022-06-03T14:20:53Z")

</div>

Alright…even after I reinstalled and verified the following. After going back and recleaning/sampling my data to a smaller subset, I finally tried to rerun my model and the kernel still died.

I uninstalled everything and installed using this page [Installation Guide (Linux) · pymc-devs/pymc Wiki (github.com)](https://github.com/pymc-devs/pymc/wiki/Installation-Guide-(Linux)) that @cluhmann suggested. Using the commands `pip install pymc --pre` installs aesara version 2.6.2.

When I upgraded aesara manually to version 2.6.6 as stated in the requirements page reference above, 2.6.2 is installed.

When I install the latest from GIT, the right versions seem to be installed, but the kernel still dies. Is there a way to flash any errors I may not be seeing?

When I run as a script through the command line terminal, I just see, “segmentation fault” appear.

---

<div class="post-metadata">

**Author:** ![cluhmann](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/cluhmann/32/3083_2.png) [@cluhmann](https://discourse.pymc.io/u/cluhmann)\
**Post date:** [June 4, 2022, 4:23pm UTC](https://discourse.pymc.io/t/kernel-is-now-dying-with-on-a-model-that-ran-before/9525/9 "2022-06-04T16:23:47Z")

</div>

The kernel stopping is probably due to the segmentation fault you’re seeing in the terminal. Can you share the full message that is generated when it faults?

---

<div class="post-metadata">

**Author:** ![jordan.howell2](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/jordan.howell2/32/1742_2.png) [@jordan.howell2](https://discourse.pymc.io/u/jordan.howell2)\
**Post date:** [June 4, 2022, 8:32pm UTC](https://discourse.pymc.io/t/kernel-is-now-dying-with-on-a-model-that-ran-before/9525/10 "2022-06-04T20:32:53Z")

</div>

That’s the entire message it shows….segmentation fault. Is there a setting I can set for it to print out the entire message? I was also told offline of the discourse that it may be a shape issue. I need to check that.

 ![CBBA9C877ECB4AEFB329CA4716C38C09.png](https://canada1.discourse-cdn.com/flex036/uploads/pymc3/original/2X/f/ffeee1ddf87037f94dd158f88fd9ced464f4be34.png)

---

<div class="post-metadata">

**Author:** ![cluhmann](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/cluhmann/32/3083_2.png) [@cluhmann](https://discourse.pymc.io/u/cluhmann)\
**Post date:** [June 4, 2022, 9:36pm UTC](https://discourse.pymc.io/t/kernel-is-now-dying-with-on-a-model-that-ran-before/9525/11 "2022-06-04T21:36:55Z")

</div>

Sorry, I intended to ask about whatever output precedes the fault message. That might provide some indication about where along the way the fault is occurring. Is it during model construction? Is it during sampling? Etc.

---

<div class="post-metadata">

**Author:** ![jordan.howell2](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/jordan.howell2/32/1742_2.png) [@jordan.howell2](https://discourse.pymc.io/u/jordan.howell2)\
**Post date:** [June 4, 2022, 10:00pm UTC](https://discourse.pymc.io/t/kernel-is-now-dying-with-on-a-model-that-ran-before/9525/12 "2022-06-04T22:00:37Z")

</div>

Sampling

Jordan

---

<div class="post-metadata">

**Author:** ![cluhmann](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/cluhmann/32/3083_2.png) [@cluhmann](https://discourse.pymc.io/u/cluhmann)\
**Post date:** [June 4, 2022, 10:26pm UTC](https://discourse.pymc.io/t/kernel-is-now-dying-with-on-a-model-that-ran-before/9525/13 "2022-06-04T22:26:30Z")

</div>

Does sampling begin? How far does it get? Does it always fail at the same place during sampling or does it vary each time you run it?

---

<div class="post-metadata">

**Author:** ![jordan.howell2](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/jordan.howell2/32/1742_2.png) [@jordan.howell2](https://discourse.pymc.io/u/jordan.howell2)\
**Post date:** [June 5, 2022, 1:16am UTC](https://discourse.pymc.io/t/kernel-is-now-dying-with-on-a-model-that-ran-before/9525/14 "2022-06-05T01:16:15Z")

</div>

Ahh! My apologies. The sampling begins and shows “compiling” on the screen. That’s as far as it gets before it dies.

Jordan

---

<div class="post-metadata">

**Author:** ![jordan.howell2](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.pymc.io/jordan.howell2/32/1742_2.png) [@jordan.howell2](https://discourse.pymc.io/u/jordan.howell2)\
**Post date:** [June 7, 2022, 12:40pm UTC](https://discourse.pymc.io/t/kernel-is-now-dying-with-on-a-model-that-ran-before/9525/15 "2022-06-07T12:40:21Z")

</div>

Update:

I’ve updated aesara. I’m now working with:  
PyMC Version: 4.0.0b6  
Aesara Version: 2.7.1  
Arvize Verions: 0.12.1

I am experimenting on a simple model:

```auto
with pm.Model() as model:
    
    a = pm.Normal('base_sales', 0, 1)
    
    likelihood = pm.ZeroInflatedPoisson('y_hat',
                           mu = a,
                           psi = .01,
                           observed = y)

    trace = pymc.sampling_jax.sample_numpyro_nuts(tune=1000, draws = 2000)

```

This now gets past the sampling before the kernel dies.

This is the last message I get before the kernel dies.

```auto
Sampling time = 0:03:05.404196
Transforming variables...
Transformation time = 0:00:00.009645

```

When I change the same model to sample using `trace = pm.sample(tune=2000, draws = 1000)`,  
the kernel did not die but gave the following warning:

```auto
Sampling 4 chains for 1_000 tune and 2_000 draw iterations (4_000 + 8_000 draws total) took 633 seconds.
The acceptance probability does not match the target. It is 0.9035, but should be close to 0.8. Try to increase the number of tuning steps.

```

I’m not sure what `acceptance probability` means but could that have something to do with the kernel dieing when JAX is used?

When I increase the tuning size with Jax, it didn’t die. Is this a coincidence or does it make sense that acceptance probability (whatever that is) is what is causing the kernel to die?