Environment not working anymore on macos

1

Hi,

I’ve been working with the same conda environment for weeks and suddenly my models don’t compile anymore.
I’m not sure what changed, I’m still on python 3.11.8, pymc 5.10.2, pytensor 2.18.6 as I’ve been for weeks.
I’m on macos on apple silicon.

Everytime I try to compile a model I get this error message printed in a loop

ERROR (pytensor.graph.rewriting.basic): Rewrite failure due to: constant_folding
ERROR (pytensor.graph.rewriting.basic): node: ExpandDims{axis=0}(2.0)
ERROR (pytensor.graph.rewriting.basic): TRACEBACK:
ERROR (pytensor.graph.rewriting.basic): Traceback (most recent call last):
  File "/opt/homebrew/Caskroom/miniforge/base/envs/pymc_env/lib/python3.11/site-packages/pytensor/graph/rewriting/basic.py", line 1922, in process_node
    replacements = node_rewriter.transform(fgraph, node)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/miniforge/base/envs/pymc_env/lib/python3.11/site-packages/pytensor/graph/rewriting/basic.py", line 1082, in transform
    return self.fn(fgraph, node)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/miniforge/base/envs/pymc_env/lib/python3.11/site-packages/pytensor/tensor/rewriting/basic.py", line 1105, in constant_folding
    thunk = node.op.make_thunk(node, storage_map, compute_map, no_recycling=[])
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/miniforge/base/envs/pymc_env/lib/python3.11/site-packages/pytensor/link/c/op.py", line 119, in make_thunk
    return self.make_c_thunk(node, storage_map, compute_map, no_recycling)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/miniforge/base/envs/pymc_env/lib/python3.11/site-packages/pytensor/link/c/op.py", line 84, in make_c_thunk
    outputs = cl.make_thunk(
              ^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/miniforge/base/envs/pymc_env/lib/python3.11/site-packages/pytensor/link/c/basic.py", line 1209, in make_thunk
    cthunk, module, in_storage, out_storage, error_storage = self.__compile__(
                                                             ^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/miniforge/base/envs/pymc_env/lib/python3.11/site-packages/pytensor/link/c/basic.py", line 1129, in __compile__
    thunk, module = self.cthunk_factory(
                    ^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/miniforge/base/envs/pymc_env/lib/python3.11/site-packages/pytensor/link/c/basic.py", line 1653, in cthunk_factory
    module = cache.module_from_key(key=key, lnk=self)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/miniforge/base/envs/pymc_env/lib/python3.11/site-packages/pytensor/link/c/cmodule.py", line 1231, in module_from_key
    module = lnk.compile_cmodule(location)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/miniforge/base/envs/pymc_env/lib/python3.11/site-packages/pytensor/link/c/basic.py", line 1552, in compile_cmodule
    module = c_compiler.compile_str(
             ^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/miniforge/base/envs/pymc_env/lib/python3.11/site-packages/pytensor/link/c/cmodule.py", line 2640, in compile_str
    raise CompileError(
pytensor.link.c.exceptions.CompileError: Compilation failed (return status=1):
/opt/homebrew/Caskroom/miniforge/base/envs/pymc_env/bin/clang++ -dynamiclib -g -O3 -fno-math-errno -Wno-unused-label -Wno-unused-variable -Wno-write-strings -Wno-c++11-narrowing -fno-exceptions -fno-unwind-tables -fno-asynchronous-unwind-tables -DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION -fPIC -undefined dynamic_lookup -I/opt/homebrew/Caskroom/miniforge/base/envs/pymc_env/lib/python3.11/site-packages/numpy/core/include -I/opt/homebrew/Caskroom/miniforge/base/envs/pymc_env/include/python3.11 -I/opt/homebrew/Caskroom/miniforge/base/envs/pymc_env/lib/python3.11/site-packages/pytensor/link/c/c_code -L/opt/homebrew/Caskroom/miniforge/base/envs/pymc_env/lib -fvisibility=hidden -o /Users/ff/.pytensor/compiledir_macOS-14.4-arm64-arm-64bit-arm-3.11.8-64/tmpm89mw6ki/mfeba4d434591def33153ec442edc27a735fd9fa46fd5f1fae8efd4f490539b0a.so /Users/ff/.pytensor/compiledir_macOS-14.4-arm64-arm-64bit-arm-3.11.8-64/tmpm89mw6ki/mod.cpp
In file included from /Users/ff/.pytensor/compiledir_macOS-14.4-arm64-arm-64bit-arm-3.11.8-64/tmpm89mw6ki/mod.cpp:1:
In file included from /opt/homebrew/Caskroom/miniforge/base/envs/pymc_env/include/python3.11/Python.h:23:
/opt/homebrew/Caskroom/miniforge/base/envs/pymc_env/bin/../include/c++/v1/stdlib.h:150:34: error: unknown type name 'ldiv_t'
inline _LIBCPP_INLINE_VISIBILITY ldiv_t div(long __x, long __y) _NOEXCEPT {
                                 ^
/opt/homebrew/Caskroom/miniforge/base/envs/pymc_env/bin/../include/c++/v1/stdlib.h:151:12: error: no member named 'ldiv' in the global namespace
  return ::ldiv(__x, __y);
         ~~^
/opt/homebrew/Caskroom/miniforge/base/envs/pymc_env/bin/../include/c++/v1/stdlib.h:154:34: error: unknown type name 'lldiv_t'
inline _LIBCPP_INLINE_VISIBILITY lldiv_t div(long long __x,
                                 ^
/opt/homebrew/Caskroom/miniforge/base/envs/pymc_env/bin/../include/c++/v1/stdlib.h:156:12: error: no member named 'lldiv' in the global namespace
  return ::lldiv(__x, __y);
         ~~^
In file included from /Users/ff/.pytensor/compiledir_macOS-14.4-arm64-arm-64bit-arm-3.11.8-64/tmpm89mw6ki/mod.cpp:1:
In file included from /opt/homebrew/Caskroom/miniforge/base/envs/pymc_env/include/python3.11/Python.h:26:
/opt/homebrew/Caskroom/miniforge/base/envs/pymc_env/bin/../include/c++/v1/string.h:95:102: error: unknown type name 'size_t'
inline _LIBCPP_HIDE_FROM_ABI _LIBCPP_PREFERRED_OVERLOAD const void* memchr(const void* __s, int __c, size_t __n) {
                                                                                                     ^
/opt/homebrew/Caskroom/miniforge/base/envs/pymc_env/bin/../include/c++/v1/string.h:98:90: error: unknown type name 'size_t'
inline _LIBCPP_HIDE_FROM_ABI _LIBCPP_PREFERRED_OVERLOAD void* memchr(void* __s, int __c, size_t __n) {
                                                                                         ^
In file included from /Users/ff/.pytensor/compiledir_macOS-14.4-arm64-arm-64bit-arm-3.11.8-64/tmpm89mw6ki/mod.cpp:1:
/opt/homebrew/Caskroom/miniforge/base/envs/pymc_env/include/python3.11/Python.h:29:12: fatal error: 'unistd.h' file not found
#  include <unistd.h>
           ^~~~~~~~~~
7 errors generated.

I tried to create a new environment and install pymc using the installation commands

conda create -c conda-forge -n pymc_env "pymc>=5"
conda activate pymc_env

and my code still produce the same errors in the new environment.

Is there anybody on macos who experiences something similar or has any fix for this?

EDIT: this only seems to be an issue when using a notebook within Visual Studio Code. Jupyter Lab works fine at the moment.

2

This must be a path issue. I would recommend installing micro-mamba directly and not use homebrew.

3

Happened to me too, literally starting today after an overnight macOS reboot… (Yes, using VS Code)

(Using miniconda)

4

I am experiencing the same issue. Using miniconda with VSCode.

5

Does it work outside of vscode? Directly in a terminal with environment activated?

VSCode is rather famous for breaking things from one release to another and you are all pointing it out (although everyone uses vscode so it’s not a very sensitive measure :))

6

My environment (python 3.11.8, pymc 5.12.0) is currently working fine in jupyter lab and when running code in the terminal as a .py script.

I don’t remember this problem starting right after any specific vscode update - but maybe it’s one of its extensions autoupdating.
I’ve had a look at vscode’s settings and I don’t see any changes to environment variables/paths or clang/C/C++ related stuff.

I tried to install micromamba without going through homebrew but my new environments were still saved under the homebrew/miniforge envs directory and were having the same issues.
I don’t have time to nuke my homebrew/conda setup at the moment, but I’ll see if new versions of vscode fix this.

7

I’m running into the same issue and filed an issue here: BUG: Compilation fails on OSX · Issue #991 · pymc-devs/pytensor · GitHub

8

For some reason it was using clang++ from a different conda environment. Setting it manually fixed the issue. But not sure why it’s not selecting clang++ from the currently active env.

9

How did you set it manually?

10

pytensor.config.cxx = "path_to_clang++"
But you really shouldn’t have to do that.

11

I know but my ENV also stopped working and I’ve tried everything, including rebuilding the ENV from scratch and nothing…

12

Just updated Macos and new xcode, pymc/pytensor broken and cannot fix.

13

@maresb have you heard of any conda issues with packages that require C compilers in the new macOS?

14

Also CC @lucianopaz

15

This seems to be a VS code issue. It works when I use Positron.

16

Yup. Works on Dataspell and Jupyter Notebook/Lab. So a VS Code issue on my side.

17

Removing CommandLineTools and re-installing it worked for me after my Mac update. Anyone else?

sudo rm -rf /Library/Developer/CommandLineTools
sudo xcode-select --install

Pytensor has the below config for me

pytensor.config.cxx = '/usr/bin/clang++'

setting it to below runs the python backend I believe which is another workaround

pytensor.config.cxx = ''
18

I’ve been dealing with similar issues on my end.

The first part of the block doesn’t seem to help with my particular issue.

sudo rm -rf /Library/Developer/CommandLineTools
sudo xcode-select --install

However, running via VScode setting pytensor.config.cxx = '' will now let my script sample - although very slowly. For some reason pointing to the clang++ path in my conda environment does not help (which may be the reason for the xcode/commandlinetools reinstall.

19

This works in VS Code. But definitely slower due to:

UserWarning: Loop fusion failed because the resulting node would exceed the kernel argument limit.
20

I see, working on my own issue where pytensor/pymc hangs in different spots based on flags, and I was able to get pm.sample() to work using the code below.

pytensor.config.gcc__cxxflags = '-L/opt/miniconda3/envs/pymc_env/lib -march=native'

So that its clear what I have done. It seems a potential solution to my variation of ‘pymc environment not working anymore on macos’ seems to be the following:

  1. set the BLAS flag as an environment variable in your conda environment, then
conda env config vars set PYTENSOR_FLAGS="blas__ldflags=-framework Accelerate"
  1. set a pytensor flag in your python script in vscode as follows:
pytensor.config.gcc__cxxflags = '-L/opt/miniconda3/envs/pymc_env/lib -march=native'

I no longer need to set pytensor.config.cxx = '' it seems? Apologies for not being so technically minded, I am not a developer and very new to this library. Note the paths I have here are for my installed environment, your environment name might be different or the path to your environment/ conda might be different.

1 Like