Wierd problem installing PYMC

Questions v5
1

I was just starting a new project, like I normally do, and tried installing pymc like this:

conda create -n multinom3 python=3.12 pymc=5.21.0 arviz jupyter -c conda-forge

This is on windows 11.
Note I pinned these versions only in later experiments when I could not get defaults to work either.
However when I try to run a simple model with this (any model) I get errors:

import pymc as pm
import numpy as np
import arviz as az

# Generate some fake data
np.random.seed(123)
data = np.random.normal(loc=5.0, scale=2.0, size=100)

with pm.Model() as model:
    mu = pm.Normal("mu", mu=0, sigma=10)
    sigma = pm.HalfNormal("sigma", sigma=10)
    observed = pm.Normal("observed", mu=mu, sigma=sigma, observed=data)
    trace = pm.sample(1000, tune=1000, cores =1)

Initializing NUTS using jitter+adapt_diag…
ERROR (pytensor.graph.rewriting.basic): Rewrite failure due to: constant_folding
ERROR (pytensor.graph.rewriting.basic): node: Cast{float64}(0)
ERROR (pytensor.graph.rewriting.basic): TRACEBACK:
ERROR (pytensor.graph.rewriting.basic): Traceback (most recent call last)
ETC ETC ETC

I tried various things like downgrading numpy but I just cannot get things to work. What is even stranger is that my existing environments I have used for other projects are working just fine using these same version numbers.

I also tried exporting an existing environment and recreating it from that (with everything pinned) and that does work. I am mystified.

Comparing exported environments, maybe this difference is what matters?

older working environment:

 - vc=14.3=hbf610ac_24
 - vc14_runtime=14.42.34438=hfd919c2_24
 - vs2015_runtime=14.42.34438=h7142326_24

vs in broken environment

  - vc=14.3=h2b53caa_26
  - vc14_runtime=14.42.34438=hfd919c2_26
  - vs2015_runtime=14.42.34438=h7142326_26

I.e. the 24 vs the 26? I am just grasping at straws here though. Anyone else run into something like this?

2

Can you post versions for pymc and pytensor? (or just confirm they are both the latest releases as shown on the github page)

Edit: Just noticed you’re version pinning pymc – is that necessary? You might need an associated pin on pytensor.

3

The pins are not necessary, I was just ‘trying things’ to get it to work. Pytensor version was 2.28.2 (I also tried 2.28.3)

4

Just to clarify I also tried to create a fresh environment following the instructions on the pymc website and got the same errors. (with default versions for all) . I tried many things lol, and one thing I found that definitely worked is switching over to my MacBook for the time being :wink:

5

By the way, I don’t think those differences you posted should be the determining factor. Those look like C++ runtimes, but your error is happening at the pytensor level, before the logp function is compiled.

6

Ok thanks! Here is more of the error message if it helps:

Initializing NUTS using jitter+adapt_diag...
ERROR (pytensor.graph.rewriting.basic): Rewrite failure due to: constant_folding
ERROR (pytensor.graph.rewriting.basic): node: Cast{float64}(0)
ERROR (pytensor.graph.rewriting.basic): TRACEBACK:
ERROR (pytensor.graph.rewriting.basic): Traceback (most recent call last):
  File "c:\Users\rjljr\miniconda3\envs\multinom\Lib\site-packages\pytensor\graph\rewriting\basic.py", line 1922, in process_node
    replacements = node_rewriter.transform(fgraph, node)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "c:\Users\rjljr\miniconda3\envs\multinom\Lib\site-packages\pytensor\graph\rewriting\basic.py", line 1086, in transform
    return self.fn(fgraph, node)
           ^^^^^^^^^^^^^^^^^^^^^
  File "c:\Users\rjljr\miniconda3\envs\multinom\Lib\site-packages\pytensor\tensor\rewriting\basic.py", line 1173, in constant_folding
    return unconditional_constant_folding.transform(fgraph, node)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "c:\Users\rjljr\miniconda3\envs\multinom\Lib\site-packages\pytensor\graph\rewriting\basic.py", line 1086, in transform
    return self.fn(fgraph, node)
           ^^^^^^^^^^^^^^^^^^^^^
  File "c:\Users\rjljr\miniconda3\envs\multinom\Lib\site-packages\pytensor\tensor\rewriting\basic.py", line 1122, in unconditional_constant_folding
    thunk = node.op.make_thunk(node, storage_map, compute_map, no_recycling=[])
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "c:\Users\rjljr\miniconda3\envs\multinom\Lib\site-packages\pytensor\link\c\op.py", line 119, in make_thunk
    return self.make_c_thunk(node, storage_map, compute_map, no_recycling)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "c:\Users\rjljr\miniconda3\envs\multinom\Lib\site-packages\pytensor\link\c\op.py", line 84, in make_c_thunk
    outputs = cl.make_thunk(
              ^^^^^^^^^^^^^^
  File "c:\Users\rjljr\miniconda3\envs\multinom\Lib\site-packages\pytensor\link\c\basic.py", line 1185, in make_thunk
    cthunk, module, in_storage, out_storage, error_storage = self.__compile__(
                                                             ^^^^^^^^^^^^^^^^^
  File "c:\Users\rjljr\miniconda3\envs\multinom\Lib\site-packages\pytensor\link\c\basic.py", line 1102, in __compile__
    thunk, module = self.cthunk_factory(
                    ^^^^^^^^^^^^^^^^^^^^
  File "c:\Users\rjljr\miniconda3\envs\multinom\Lib\site-packages\pytensor\link\c\basic.py", line 1626, in cthunk_factory
    module = cache.module_from_key(key=key, lnk=self)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "c:\Users\rjljr\miniconda3\envs\multinom\Lib\site-packages\pytensor\link\c\cmodule.py", line 1250, in module_from_key
    module = lnk.compile_cmodule(location)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "c:\Users\rjljr\miniconda3\envs\multinom\Lib\site-packages\pytensor\link\c\basic.py", line 1527, in compile_cmodule
    module = c_compiler.compile_str(
             ^^^^^^^^^^^^^^^^^^^^^^^
  File "c:\Users\rjljr\miniconda3\envs\multinom\Lib\site-packages\pytensor\link\c\cmodule.py", line 2688, in compile_str
    return dlimport(lib_filename)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "c:\Users\rjljr\miniconda3\envs\multinom\Lib\site-packages\pytensor\link\c\cmodule.py", line 346, in dlimport
    rval = __import__(module_name, {}, {}, [module_name])
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ImportError: DLL load failed while importing m300014ab234446dc184b72b5e07d4663d2813454bb33582d49f5e173c77f73aa: The specified procedure could not be found.

This repeats over and over for with a different (hash?) string for the 'specified procedure` and different ops.

7

If there are other tests you can think of that i could try, I would appreciate it!

8

Well now I do think those differences could be relevant. Just to rule it out, could you clear your pytensor cache (pytensor-cache clear )

9

Ok tried that, still same errors. also did purge just to be sure.

10

The compiler / path machinery is failing, may be a general incompatibility with a recent version of windows 11? Don’t know if other folks out there have succeeded installing PyMC on recent Windows 11.

Perhaps @michaelosthege ?

11

In the meantime you can try to avoid the C backend. Set pytensor.config.cxx="" and for sampling with PyMC use the numba backend by passing compile_kwargs={mode:"NUMBA"} or use the nutpie sampler which also defaults to numba.

12

Thanks this workaround (and using nutpie) works great!
Today I will try to find time to find the minimal diff between my working and not working environments and report back :slight_smile:

13

Comparing conda exports from working and not working environments, I found that pinning libgcc to an earlier version fixes it:

This envirnoment works:

name: working_env
channels:
  - conda-forge
  - defaults
dependencies:
  - python = 3.12
  - scikit-learn
  - pandas
  - numpy
  - matplotlib
  - pytensor=2.28.2
  - pymc = 5.21
  - arviz = 0.21
  - jupyter
  - pip
  - libgcc=14.2.0=h1383e82_2

This one doesn’t (same as above except remove the -libgcc pin:

name: broken_env
channels:
  - conda-forge
  - defaults
dependencies:
  - python = 3.12
  - scikit-learn
  - pandas
  - numpy
  - matplotlib
  - pytensor=2.28.2
  - pymc = 5.21
  - arviz = 0.21
  - jupyter
  - pip

In this version, libgcc is 15.1.0

The other pinned versions may not matter, I just have not had a chance to test that (building and destroying conda environments is a bit time consuming !)

2 Likes
14

CC @maresb

15

I did test removing all the pins, and it is enough to just add -libgcc=14.2 to get things to work with no other pinning.

1 Like
16

Thanks @DrEntropy for the diagnosis!

Note that libgcc 15 came out 3 days ago on conda-forge: Files | Anaconda.org

I’m not sure what we need to do to stabilize this, or if we’ll have this problem on other platforms? Do we need to pin libgcc<15 for everyone before we release a new version? Also, I thought gcc was supposed to be forwards-compatible.

17

Tracking this in GitHub in Pytensor finds wrong compiler on windows when `libgcc>15` is installed · Issue #1398 · pymc-devs/pytensor · GitHub

2 Likes
18

Thanks, for anyone else reading this, it was fixed with Remove iostream include for C backend by maresb · Pull Request #1400 · pymc-devs/pytensor . Thanks @maresb !

1 Like