Having issues with pymc on databricks cloud platform

Questions v5
1

Any other Databricks users experiencing the same? A colleague of mine noticed a couple days ago that our clusters were failing to install pymc. We found a workaround by adding an init script like below, which seemed to fix the install failures.

pip install --upgrade 'pip>=25.0' 'setuptools>=77.0.0' 'wheel>=0.43.0'

However, none of my models will finish sampling with the workaround.
For reference, it is a Databricks 16.4 LTS cluster running python 3.12.3 with the following libraries

image
image1065×578 21.5 KB

A bit odd because everything was going smoothly for several months up until Friday, and I didn’t change anything with my cluster before it stopped working.

2

Unfortunately I don’t have a solution, but I am having the same issue. I’m also using Databricks Runtime 16.4 LTS, python 3.12.3. No additional libraries on my end.

I tried using your init script but had no luck installing pymc.

I’ll post an update if I make any progress. Good luck!

3

It’s due to miniKanren 1.0.4 release requires setuptools >= 77.0.0 · Issue #1495 · pymc-devs/pytensor · GitHub

Should go away with the latest PyTensor release we did today

1 Like