Pytensor compilation error

Questions version agnostic
,
1

Hi, I just updated PyMC to v5 and few updates of my system (Mac OS), now I’m getting this error while setting testing the PyMC environment. Does anyone has experienced same thing?

import pymc as pm
import pytensor
from pytensor import tensor as pt

print(f"Running on PyMC v{pm.__version__}")

WARNING (pytensor.tensor.blas): Using NumPy C-API based implementation for BLAS functions.
Running on PyMC v5.0.0

with pm.Model() as model:
    mu = pm.Normal('mu')
    log_sigma = pm.Normal('log_sigma')
    y = pm.Normal('y', mu=mu, sigma=pt.exp(log_sigma), observed=[4,5,6])

Output exceeds the size limit. Open the full output data in a text editor
ERROR (pytensor.graph.rewriting.basic): Rewrite failure due to: constant_folding
ERROR (pytensor.graph.rewriting.basic): node: InplaceDimShuffle{}(TensorConstant{(1,) of 3})
ERROR (pytensor.graph.rewriting.basic): TRACEBACK:
ERROR (pytensor.graph.rewriting.basic): Traceback (most recent call last):
  File "/Users/marcelino/miniconda3/envs/pymc_env/lib/python3.9/site-packages/pytensor/graph/rewriting/basic.py", line 1933, in process_node
    replacements = node_rewriter.transform(fgraph, node)
  File "/Users/marcelino/miniconda3/envs/pymc_env/lib/python3.9/site-packages/pytensor/graph/rewriting/basic.py", line 1092, in transform
    return self.fn(fgraph, node)
  File "/Users/marcelino/miniconda3/envs/pymc_env/lib/python3.9/site-packages/pytensor/tensor/rewriting/basic.py", line 1141, in constant_folding
    thunk = node.op.make_thunk(node, storage_map, compute_map, no_recycling=[])
  File "/Users/marcelino/miniconda3/envs/pymc_env/lib/python3.9/site-packages/pytensor/link/c/op.py", line 131, in make_thunk
    return self.make_c_thunk(node, storage_map, compute_map, no_recycling)
  File "/Users/marcelino/miniconda3/envs/pymc_env/lib/python3.9/site-packages/pytensor/link/c/op.py", line 96, in make_c_thunk
    outputs = cl.make_thunk(
  File "/Users/marcelino/miniconda3/envs/pymc_env/lib/python3.9/site-packages/pytensor/link/c/basic.py", line 1202, in make_thunk
    cthunk, module, in_storage, out_storage, error_storage = self.__compile__(
  File "/Users/marcelino/miniconda3/envs/pymc_env/lib/python3.9/site-packages/pytensor/link/c/basic.py", line 1122, in __compile__
    thunk, module = self.cthunk_factory(
  File "/Users/marcelino/miniconda3/envs/pymc_env/lib/python3.9/site-packages/pytensor/link/c/basic.py", line 1647, in cthunk_factory
    module = cache.module_from_key(key=key, lnk=self)
  File "/Users/marcelino/miniconda3/envs/pymc_env/lib/python3.9/site-packages/pytensor/link/c/cmodule.py", line 1229, in module_from_key
    module = lnk.compile_cmodule(location)
  File "/Users/marcelino/miniconda3/envs/pymc_env/lib/python3.9/site-packages/pytensor/link/c/basic.py", line 1546, in compile_cmodule
    module = c_compiler.compile_str(
  File "/Users/marcelino/miniconda3/envs/pymc_env/lib/python3.9/site-packages/pytensor/link/c/cmodule.py", line 2641, in compile_str
...
ld: library not found for -lSystem
clang-14: error: linker command failed with exit code 1 (use -v to see invocation)



You can find the C code in this temporary file: /var/folders/nv/ntvjypdj4tqbgznkrcfhzm240000gn/T/pytensor_compilation_error_osgnqwmm
library System is not found.
Output exceeds the size limit. Open the full output data in a text editor
ERROR (pytensor.graph.rewriting.basic): Rewrite failure due to: constant_folding
ERROR (pytensor.graph.rewriting.basic): node: InplaceDimShuffle{}(TensorConstant{(1,) of 3})
ERROR (pytensor.graph.rewriting.basic): TRACEBACK:
ERROR (pytensor.graph.rewriting.basic): Traceback (most recent call last):
  File "/Users/marcelino/miniconda3/envs/pymc_env/lib/python3.9/site-packages/pytensor/graph/rewriting/basic.py", line 1933, in process_node
    replacements = node_rewriter.transform(fgraph, node)
  File "/Users/marcelino/miniconda3/envs/pymc_env/lib/python3.9/site-packages/pytensor/graph/rewriting/basic.py", line 1092, in transform
    return self.fn(fgraph, node)
  File "/Users/marcelino/miniconda3/envs/pymc_env/lib/python3.9/site-packages/pytensor/tensor/rewriting/basic.py", line 1141, in constant_folding
    thunk = node.op.make_thunk(node, storage_map, compute_map, no_recycling=[])
  File "/Users/marcelino/miniconda3/envs/pymc_env/lib/python3.9/site-packages/pytensor/link/c/op.py", line 131, in make_thunk
    return self.make_c_thunk(node, storage_map, compute_map, no_recycling)
  File "/Users/marcelino/miniconda3/envs/pymc_env/lib/python3.9/site-packages/pytensor/link/c/op.py", line 96, in make_c_thunk
    outputs = cl.make_thunk(
  File "/Users/marcelino/miniconda3/envs/pymc_env/lib/python3.9/site-packages/pytensor/link/c/basic.py", line 1202, in make_thunk
    cthunk, module, in_storage, out_storage, error_storage = self.__compile__(
  File "/Users/marcelino/miniconda3/envs/pymc_env/lib/python3.9/site-packages/pytensor/link/c/basic.py", line 1122, in __compile__
    thunk, module = self.cthunk_factory(
  File "/Users/marcelino/miniconda3/envs/pymc_env/lib/python3.9/site-packages/pytensor/link/c/basic.py", line 1647, in cthunk_factory
    module = cache.module_from_key(key=key, lnk=self)
  File "/Users/marcelino/miniconda3/envs/pymc_env/lib/python3.9/site-packages/pytensor/link/c/cmodule.py", line 1229, in module_from_key
    module = lnk.compile_cmodule(location)
  File "/Users/marcelino/miniconda3/envs/pymc_env/lib/python3.9/site-packages/pytensor/link/c/basic.py", line 1546, in compile_cmodule
    module = c_compiler.compile_str(
  File "/Users/marcelino/miniconda3/envs/pymc_env/lib/python3.9/site-packages/pytensor/link/c/cmodule.py", line 2641, in compile_str
...
ld: library not found for -lSystem
clang-14: error: linker command failed with exit code 1 (use -v to see invocation)



You can find the C code in this temporary file: /var/folders/nv/ntvjypdj4tqbgznkrcfhzm240000gn/T/pytensor_compilation_error_fgg5rho8
library System is not found.
Output exceeds the size limit. Open the full output data in a text editor
ERROR (pytensor.graph.rewriting.basic): Rewrite failure due to: constant_folding
ERROR (pytensor.graph.rewriting.basic): node: InplaceDimShuffle{}(TensorConstant{(1,) of 3})
ERROR (pytensor.graph.rewriting.basic): TRACEBACK:
ERROR (pytensor.graph.rewriting.basic): Traceback (most recent call last):
  File "/Users/marcelino/miniconda3/envs/pymc_env/lib/python3.9/site-packages/pytensor/graph/rewriting/basic.py", line 1933, in process_node
    replacements = node_rewriter.transform(fgraph, node)
  File "/Users/marcelino/miniconda3/envs/pymc_env/lib/python3.9/site-packages/pytensor/graph/rewriting/basic.py", line 1092, in transform
    return self.fn(fgraph, node)
  File "/Users/marcelino/miniconda3/envs/pymc_env/lib/python3.9/site-packages/pytensor/tensor/rewriting/basic.py", line 1141, in constant_folding
    thunk = node.op.make_thunk(node, storage_map, compute_map, no_recycling=[])
  File "/Users/marcelino/miniconda3/envs/pymc_env/lib/python3.9/site-packages/pytensor/link/c/op.py", line 131, in make_thunk
    return self.make_c_thunk(node, storage_map, compute_map, no_recycling)
  File "/Users/marcelino/miniconda3/envs/pymc_env/lib/python3.9/site-packages/pytensor/link/c/op.py", line 96, in make_c_thunk
    outputs = cl.make_thunk(
  File "/Users/marcelino/miniconda3/envs/pymc_env/lib/python3.9/site-packages/pytensor/link/c/basic.py", line 1202, in make_thunk
    cthunk, module, in_storage, out_storage, error_storage = self.__compile__(
  File "/Users/marcelino/miniconda3/envs/pymc_env/lib/python3.9/site-packages/pytensor/link/c/basic.py", line 1122, in __compile__
    thunk, module = self.cthunk_factory(
  File "/Users/marcelino/miniconda3/envs/pymc_env/lib/python3.9/site-packages/pytensor/link/c/basic.py", line 1647, in cthunk_factory
    module = cache.module_from_key(key=key, lnk=self)
  File "/Users/marcelino/miniconda3/envs/pymc_env/lib/python3.9/site-packages/pytensor/link/c/cmodule.py", line 1229, in module_from_key
    module = lnk.compile_cmodule(location)
  File "/Users/marcelino/miniconda3/envs/pymc_env/lib/python3.9/site-packages/pytensor/link/c/basic.py", line 1546, in compile_cmodule
    module = c_compiler.compile_str(
  File "/Users/marcelino/miniconda3/envs/pymc_env/lib/python3.9/site-packages/pytensor/link/c/cmodule.py", line 2641, in compile_str
...
ld: library not found for -lSystem
clang-14: error: linker command failed with exit code 1 (use -v to see invocation)



You can find the C code in this temporary file: /var/folders/nv/ntvjypdj4tqbgznkrcfhzm240000gn/T/pytensor_compilation_error_rvr4bkw9
library System is not found.
2

Are you using vscode? It seems that I also encounter the same problem.

3

How did you install PyMC?

4

Yes, I’m using vscode.

5

Basically this way:

 conda install 'pymc>=5.0.0' 'pymc-base>=5.0.0' pytensor pytensor-base python-dateutil jupyter -y
6

I would strongly suggest following the official installation instructions. Your method includes unnecessary/redundant packages (e.g., pytensor is a dependency of pymc, so it will necessarily get installed) and, more importantly, you have not specified conda-forge as the preferred channel. I don’t know if any of that is causing your specific problems. but it’s always good practice.

1 Like
7

I am having nearly the exact same error. My PyMC is in a virtual environment, installed from the instructions on the website. My PyMC version is 5.0.1, and my error when I run the same code is:

ERROR (pytensor.graph.rewriting.basic): Rewrite failure due to: constant_folding
ERROR (pytensor.graph.rewriting.basic): node: InplaceDimShuffle{}(TensorConstant{(1,) of 102})
ERROR (pytensor.graph.rewriting.basic): TRACEBACK:
ERROR (pytensor.graph.rewriting.basic): Traceback (most recent call last):
  File "/Users/joshgen/miniconda3/envs/pymc_env/lib/python3.11/site-packages/pytensor/graph/rewriting/basic.py", line 1933, in process_node
    replacements = node_rewriter.transform(fgraph, node)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/joshgen/miniconda3/envs/pymc_env/lib/python3.11/site-packages/pytensor/graph/rewriting/basic.py", line 1092, in transform
    return self.fn(fgraph, node)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/joshgen/miniconda3/envs/pymc_env/lib/python3.11/site-packages/pytensor/tensor/rewriting/basic.py", line 1141, in constant_folding
    thunk = node.op.make_thunk(node, storage_map, compute_map, no_recycling=[])
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/joshgen/miniconda3/envs/pymc_env/lib/python3.11/site-packages/pytensor/link/c/op.py", line 131, in make_thunk
    return self.make_c_thunk(node, storage_map, compute_map, no_recycling)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/joshgen/miniconda3/envs/pymc_env/lib/python3.11/site-packages/pytensor/link/c/op.py", line 96, in make_c_thunk
    outputs = cl.make_thunk(
              ^^^^^^^^^^^^^^
  File "/Users/joshgen/miniconda3/envs/pymc_env/lib/python3.11/site-packages/pytensor/link/c/basic.py", line 1202, in make_thunk
    cthunk, module, in_storage, out_storage, error_storage = self.__compile__(
                                                             ^^^^^^^^^^^^^^^^^
  File "/Users/joshgen/miniconda3/envs/pymc_env/lib/python3.11/site-packages/pytensor/link/c/basic.py", line 1122, in __compile__
    thunk, module = self.cthunk_factory(
                    ^^^^^^^^^^^^^^^^^^^^
...
              ^~~~~~~~~~
1 error generated.



You can find the C code in this temporary file: /var/folders/2k/10t0ng3n1pdd4bxjxwf5z3xw0000gn/T/pytensor_compilation_error_7lyujdty
Output exceeds the size limit. Open the full output data in a text editor
ERROR (pytensor.graph.rewriting.basic): Rewrite failure due to: constant_folding
ERROR (pytensor.graph.rewriting.basic): node: InplaceDimShuffle{}(TensorConstant{(1,) of 102})
ERROR (pytensor.graph.rewriting.basic): TRACEBACK:
ERROR (pytensor.graph.rewriting.basic): Traceback (most recent call last):
  File "/Users/joshgen/miniconda3/envs/pymc_env/lib/python3.11/site-packages/pytensor/graph/rewriting/basic.py", line 1933, in process_node
    replacements = node_rewriter.transform(fgraph, node)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/joshgen/miniconda3/envs/pymc_env/lib/python3.11/site-packages/pytensor/graph/rewriting/basic.py", line 1092, in transform
    return self.fn(fgraph, node)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/joshgen/miniconda3/envs/pymc_env/lib/python3.11/site-packages/pytensor/tensor/rewriting/basic.py", line 1141, in constant_folding
    thunk = node.op.make_thunk(node, storage_map, compute_map, no_recycling=[])
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/joshgen/miniconda3/envs/pymc_env/lib/python3.11/site-packages/pytensor/link/c/op.py", line 131, in make_thunk
    return self.make_c_thunk(node, storage_map, compute_map, no_recycling)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/joshgen/miniconda3/envs/pymc_env/lib/python3.11/site-packages/pytensor/link/c/op.py", line 96, in make_c_thunk
    outputs = cl.make_thunk(
              ^^^^^^^^^^^^^^
  File "/Users/joshgen/miniconda3/envs/pymc_env/lib/python3.11/site-packages/pytensor/link/c/basic.py", line 1202, in make_thunk
    cthunk, module, in_storage, out_storage, error_storage = self.__compile__(
                                                             ^^^^^^^^^^^^^^^^^
  File "/Users/joshgen/miniconda3/envs/pymc_env/lib/python3.11/site-packages/pytensor/link/c/basic.py", line 1122, in __compile__
    thunk, module = self.cthunk_factory(
                    ^^^^^^^^^^^^^^^^^^^^
...
              ^~~~~~~~~~
1 error generated.



You can find the C code in this temporary file: /var/folders/2k/10t0ng3n1pdd4bxjxwf5z3xw0000gn/T/pytensor_compilation_error_py60q6h0
Output exceeds the size limit. Open the full output data in a text editor
ERROR (pytensor.graph.rewriting.basic): Rewrite failure due to: constant_folding
ERROR (pytensor.graph.rewriting.basic): node: InplaceDimShuffle{}(TensorConstant{(1,) of 102})
ERROR (pytensor.graph.rewriting.basic): TRACEBACK:
ERROR (pytensor.graph.rewriting.basic): Traceback (most recent call last):
  File "/Users/joshgen/miniconda3/envs/pymc_env/lib/python3.11/site-packages/pytensor/graph/rewriting/basic.py", line 1933, in process_node
    replacements = node_rewriter.transform(fgraph, node)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/joshgen/miniconda3/envs/pymc_env/lib/python3.11/site-packages/pytensor/graph/rewriting/basic.py", line 1092, in transform
    return self.fn(fgraph, node)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/joshgen/miniconda3/envs/pymc_env/lib/python3.11/site-packages/pytensor/tensor/rewriting/basic.py", line 1141, in constant_folding
    thunk = node.op.make_thunk(node, storage_map, compute_map, no_recycling=[])
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/joshgen/miniconda3/envs/pymc_env/lib/python3.11/site-packages/pytensor/link/c/op.py", line 131, in make_thunk
    return self.make_c_thunk(node, storage_map, compute_map, no_recycling)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/joshgen/miniconda3/envs/pymc_env/lib/python3.11/site-packages/pytensor/link/c/op.py", line 96, in make_c_thunk
    outputs = cl.make_thunk(
              ^^^^^^^^^^^^^^
  File "/Users/joshgen/miniconda3/envs/pymc_env/lib/python3.11/site-packages/pytensor/link/c/basic.py", line 1202, in make_thunk
    cthunk, module, in_storage, out_storage, error_storage = self.__compile__(
                                                             ^^^^^^^^^^^^^^^^^
  File "/Users/joshgen/miniconda3/envs/pymc_env/lib/python3.11/site-packages/pytensor/link/c/basic.py", line 1122, in __compile__
    thunk, module = self.cthunk_factory(
                    ^^^^^^^^^^^^^^^^^^^^
...
              ^~~~~~~~~~
1 error generated.



You can find the C code in this temporary file: /var/folders/2k/10t0ng3n1pdd4bxjxwf5z3xw0000gn/T/pytensor_compilation_error_mdh0fve1

I am using VS Code but have tried using Jupyter notebook; I have removed the pymc environment and redownloaded, I have also deleted all pymc, theano, and pytensor from everything except that environment. Any suggestions?

8

I just got a new mac and ran into several compiler errors, including these missing standard C libraries. I had to do the following steps:

  1. I had to reinstall xcode-select (xcode-select --install, but in my case I also had to uninstall it first because I did a migration from an intel to an M1 system, so everything got messed up).

  2. BLAS doesn’t seem to grab the apple accelerate version that works on the ARM64 chips by default, so i had to do mamba install blas=*=*accelerate*

  3. I added the location of xcode CommandLineTools to the CPATH and LIBRARY_PATH globals as described here.

  4. I also encountered a problem with the -c++11-narrowing flag when compiling PyMC models, so i added the following lines into my .pytensorrc (or .aesararc):

[gcc] 
cxxflags = -Wno-c++11-narrowing

Admittedly I have no idea which of these (or which combination) are critical, but it got me up and running. If anyone can refine this down to a simple how-to for Macs, it would be great.

2 Likes
9

I wanted to add a +1 here. I am running version 5.0.1 installed via conda-forge. Running the same model as dmarcelinobr or also the intro model from Introductory Overview of PyMC — PyMC 5.0.1 documentation produces the same error from constant_folding. In the output from the compilation step, I see reports of unrecognized characters and unknown pseudo ops:

ERROR (pytensor.graph.rewriting.basic): Rewrite failure due to: constant_folding
ERROR (pytensor.graph.rewriting.basic): node: InplaceDimShuffle{}(TensorConstant{(1,) of 3})
ERROR (pytensor.graph.rewriting.basic): TRACEBACK:
ERROR (pytensor.graph.rewriting.basic): Traceback (most recent call last):
  File "/home/redacted/.conda/envs/biomarker_proxy_model/lib/python3.10/site-packages/pytensor/graph/rewriting/basic.py", line 1933, in process_node
    replacements = node_rewriter.transform(fgraph, node)
  File "/home/redacted/.conda/envs/biomarker_proxy_model/lib/python3.10/site-packages/pytensor/graph/rewriting/basic.py", line 1092, in transform
    return self.fn(fgraph, node)
  File "/home/redacted/.conda/envs/biomarker_proxy_model/lib/python3.10/site-packages/pytensor/tensor/rewriting/basic.py", line 1141, in constant_folding
    thunk = node.op.make_thunk(node, storage_map, compute_map, no_recycling=[])
  File "/home/redacted/.conda/envs/biomarker_proxy_model/lib/python3.10/site-packages/pytensor/link/c/op.py", line 131, in make_thunk
    return self.make_c_thunk(node, storage_map, compute_map, no_recycling)
  File "/home/redacted/.conda/envs/biomarker_proxy_model/lib/python3.10/site-packages/pytensor/link/c/op.py", line 96, in make_c_thunk
    outputs = cl.make_thunk(
  File "/home/redacted/.conda/envs/biomarker_proxy_model/lib/python3.10/site-packages/pytensor/link/c/basic.py", line 1202, in make_thunk
    cthunk, module, in_storage, out_storage, error_storage = self.__compile__(
  File "/home/redacted/.conda/envs/biomarker_proxy_model/lib/python3.10/site-packages/pytensor/link/c/basic.py", line 1122, in __compile__
    thunk, module = self.cthunk_factory(
  File "/home/redacted/.conda/envs/biomarker_proxy_model/lib/python3.10/site-packages/pytensor/link/c/basic.py", line 1647, in cthunk_factory
    module = cache.module_from_key(key=key, lnk=self)
  File "/home/redacted/.conda/envs/biomarker_proxy_model/lib/python3.10/site-packages/pytensor/link/c/cmodule.py", line 1231, in module_from_key
    module = lnk.compile_cmodule(location)
  File "/home/redacted/.conda/envs/biomarker_proxy_model/lib/python3.10/site-packages/pytensor/link/c/basic.py", line 1546, in compile_cmodule
    module = c_compiler.compile_str(
  File "/home/redacted/.conda/envs/biomarker_proxy_model/lib/python3.10/site-packages/pytensor/link/c/cmodule.py", line 2641, in compile_str
    raise CompileError(
pytensor.link.c.exceptions.CompileError: Compilation failed (return status=1):
/apps/easybuild/software/GCCcore/10.3.0/bin/g++ -shared -g -O3 -fno-math-errno -Wno-unused-label -Wno-unused-variable -Wno-write-strings -Wno-c++11-narrowing -fno-exceptions -fno-unwind-tables -fno-asynchronous-unwind-tables -march=skylake-avx512 -mmmx -mno-3dnow -msse -msse2 -msse3 -mssse3 -mno-sse4a -mcx16 -msahf -mmovbe -maes -mno-sha -mpclmul -mpopcnt -mabm -mno-lwp -mfma -mno-fma4 -mno-xop -mbmi -mno-sgx -mbmi2 -mno-pconfig -mno-wbnoinvd -mno-tbm -mavx -mavx2 -msse4.2 -msse4.1 -mlzcnt -mrtm -mhle -mrdrnd -mf16c -mfsgsbase -mrdseed -mprfchw -madx -mfxsr -mxsave -mxsaveopt -mavx512f -mno-avx512er -mavx512cd -mno-avx512pf -mno-prefetchwt1 -mclflushopt -mxsavec -mxsaves -mavx512dq -mavx512bw -mavx512vl -mno-avx512ifma -mno-avx512vbmi -mno-avx5124fmaps -mno-avx5124vnniw -mclwb -mno-mwaitx -mno-clzero -mpku -mno-rdpid -mno-gfni -mno-shstk -mno-avx512vbmi2 -mno-avx512vnni -mno-vaes -mno-vpclmulqdq -mno-avx512bitalg -mno-avx512vpopcntdq -mno-movdiri -mno-movdir64b -mno-waitpkg -mno-cldemote -mno-ptwrite -mno-avx512bf16 -mno-enqcmd -mno-avx512vp2intersect --param l1-cache-size=32 --param l1-cache-line-size=64 --param l2-cache-size=25344 -mtune=skylake-avx512 -DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION -m64 -fPIC -I/home/redacted/.conda/envs/biomarker_proxy_model/lib/python3.10/site-packages/numpy/core/include -I/home/redacted/.conda/envs/biomarker_proxy_model/include/python3.10 -I/home/redacted/.conda/envs/biomarker_proxy_model/lib/python3.10/site-packages/pytensor/link/c/c_code -L/home/redacted/.conda/envs/biomarker_proxy_model/lib -fvisibility=hidden -o /home/redacted/.pytensor/compiledir_Linux-3.10-el7.x86_64-x86_64-with-glibc2.17-x86_64-3.10.8-64/tmpm9qwja5n/mdb3a1a186c6e0c789bc6494774a591ac6d8ab11d3ed19bf02a4ff96ab64fa9ba.so /home/redacted/.pytensor/compiledir_Linux-3.10-el7.x86_64-x86_64-with-glibc2.17-x86_64-3.10.8-64/tmpm9qwja5n/mod.cpp -lpython3.10
/clscratch/redacted/tmp/ccTEQd3t.s: Assembler messages:
/clscratch/redacted/tmp/ccTEQd3t.s:68: Error: unknown .loc sub-directive `view'
/clscratch/redacted/tmp/ccTEQd3t.s:68: Error: junk at end of line, first unrecognized character is `-'
/clscratch/redacted/tmp/ccTEQd3t.s:70: Error: unknown .loc sub-directive `view'
/clscratch/redacted/tmp/ccTEQd3t.s:70: Error: unknown pseudo-op: `.lvu1'
/clscratch/redacted/tmp/ccTEQd3t.s:74: Error: unknown .loc sub-directive `view'
/clscratch/redacted/tmp/ccTEQd3t.s:74: Error: unknown pseudo-op: `.lvu2'
/clscratch/redacted/tmp/ccTEQd3t.s:76: Error: unknown .loc sub-directive `view'
/clscratch/redacted/tmp/ccTEQd3t.s:76: Error: unknown pseudo-op: `.lvu3'
/clscratch/redacted/tmp/ccTEQd3t.s:77: Error: unknown .loc sub-directive `view'
/clscratch/redacted/tmp/ccTEQd3t.s:77: Error: unknown pseudo-op: `.lvu4'
[...]
10

What platform are you on?

11

I’m running on Redhat 7.9 (3.10.0-1160.42.2.el7.x86_64 #1 SMP)
and below is a dump of my conda environment. Please let me know what other info might be useful. Thanks!

# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                        main  
_openmp_mutex             5.1                       1_gnu  
abseil-cpp                20211102.0           hd4dd3e8_0  
anyio                     3.5.0           py310h06a4308_0  
argon2-cffi               21.3.0             pyhd3eb1b0_0  
argon2-cffi-bindings      21.2.0          py310h7f8727e_0  
arrow                     1.2.3           py310h06a4308_0  
arrow-cpp                 8.0.0           py310h3098874_0  
arviz                     0.14.0             pyhd8ed1ab_0    conda-forge
asttokens                 2.0.5              pyhd3eb1b0_0  
attrs                     22.1.0          py310h06a4308_0  
aws-c-common              0.4.57               he6710b0_1  
aws-c-event-stream        0.1.6                h2531618_5  
aws-checksums             0.1.9                he6710b0_0  
aws-sdk-cpp               1.8.185              hce553d0_0  
babel                     2.11.0          py310h06a4308_0  
backcall                  0.2.0              pyhd3eb1b0_0  
beautifulsoup4            4.11.1          py310h06a4308_0  
binutils                  2.38                 h1680402_1  
binutils_impl_linux-64    2.38                 h2a08ee3_1  
binutils_linux-64         2.38.0               hc2dff05_0  
blas                      1.0                         mkl  
bleach                    4.1.0              pyhd3eb1b0_0  
boost-cpp                 1.73.0              h7f8727e_12  
bottleneck                1.3.5           py310ha9d4c09_0  
brotli                    1.0.9                h5eee18b_7  
brotli-bin                1.0.9                h5eee18b_7  
brotlipy                  0.7.0           py310h7f8727e_1002  
bzip2                     1.0.8                h7b6447c_0  
c-ares                    1.18.1               h7f8727e_0  
c-compiler                1.5.2                h0b41bf4_0    conda-forge
ca-certificates           2022.10.11           h06a4308_0  
cachetools                4.2.2              pyhd3eb1b0_0  
certifi                   2022.12.7       py310h06a4308_0  
cffi                      1.15.1          py310h5eee18b_3  
cftime                    1.5.1.1         py310h9102076_0  
charset-normalizer        2.0.4              pyhd3eb1b0_0  
cloudpickle               2.0.0              pyhd3eb1b0_0  
conda                     22.11.1         py310h06a4308_4  
conda-package-handling    1.9.0           py310h5eee18b_1  
cons                      0.4.5              pyhd8ed1ab_0    conda-forge
contourpy                 1.0.5           py310hdb19cb5_0  
cryptography              38.0.1          py310h9ce1e76_0  
curl                      7.86.0               h5eee18b_0  
cxx-compiler              1.5.2                hf52228f_0    conda-forge
cycler                    0.11.0             pyhd3eb1b0_0  
dbus                      1.13.18              hb2f20db_0  
debugpy                   1.5.1           py310h295c915_0  
decorator                 5.1.1              pyhd3eb1b0_0  
defusedxml                0.7.1              pyhd3eb1b0_0  
entrypoints               0.4             py310h06a4308_0  
etuples                   0.3.8              pyhd8ed1ab_0    conda-forge
executing                 0.8.3              pyhd3eb1b0_0  
expat                     2.4.9                h6a678d5_0  
fastprogress              1.0.0              pyhb85f177_0  
fftw                      3.3.9                h27cfd23_1  
filelock                  3.9.0           py310h06a4308_0  
flit-core                 3.6.0              pyhd3eb1b0_0  
fontconfig                2.14.1               h52c9d5c_1  
fonttools                 4.25.0             pyhd3eb1b0_0  
foundry_integration       0.0.6                      py_0    https://artifactory.f1.novartis.net:443/artifactory/f1-all-all-foundry-conda
freetype                  2.12.1               h4a9f257_0  
gcc                       11.2.0              h702ea55_10    conda-forge
gcc_impl_linux-64         11.2.0               h1234567_1  
gcc_linux-64              11.2.0               h5c386dc_0  
gflags                    2.2.2                he6710b0_0  
giflib                    5.2.1                h7b6447c_0  
glib                      2.69.1               he621ea3_2  
glog                      0.5.0                h2531618_0  
grpc-cpp                  1.46.1               h33aed49_0  
gst-plugins-base          1.14.0               h8213a91_2  
gstreamer                 1.14.0               h28cd5cc_2  
gxx                       11.2.0              h702ea55_10    conda-forge
gxx_impl_linux-64         11.2.0               h1234567_1  
gxx_linux-64              11.2.0               hc2dff05_0  
hdf4                      4.2.13               h3ca952b_2  
hdf5                      1.10.6               h3ffc7dd_1  
icu                       58.2                 he6710b0_3  
idna                      3.4             py310h06a4308_0  
intel-openmp              2021.4.0          h06a4308_3561  
ipykernel                 6.15.2          py310h06a4308_0  
ipython                   8.7.0           py310h06a4308_0  
ipython_genutils          0.2.0              pyhd3eb1b0_1  
jedi                      0.18.1          py310h06a4308_1  
jinja2                    3.1.2           py310h06a4308_0  
joblib                    1.1.1           py310h06a4308_0  
jpeg                      9e                   h7f8727e_0  
json5                     0.9.6              pyhd3eb1b0_0  
jsonschema                4.16.0          py310h06a4308_0  
jupyter_client            7.4.8           py310h06a4308_0  
jupyter_core              5.1.1           py310h06a4308_0  
jupyter_server            1.23.4          py310h06a4308_0  
jupyterlab                3.5.2           py310h06a4308_0  
jupyterlab_pygments       0.1.2                      py_0  
jupyterlab_server         2.16.5          py310h06a4308_0  
kernel-headers_linux-64   2.6.32              he073ed8_15    conda-forge
kiwisolver                1.4.4           py310h6a678d5_0  
krb5                      1.19.2               hac12032_0  
lcms2                     2.12                 h3be6417_0  
ld_impl_linux-64          2.38                 h1181459_1  
lerc                      3.0                  h295c915_0  
libboost                  1.73.0              h28710b8_12  
libbrotlicommon           1.0.9                h5eee18b_7  
libbrotlidec              1.0.9                h5eee18b_7  
libbrotlienc              1.0.9                h5eee18b_7  
libclang                  10.0.1          default_hb85057a_2  
libcurl                   7.86.0               h91b91d3_0  
libdeflate                1.8                  h7f8727e_5  
libedit                   3.1.20221030         h5eee18b_0  
libev                     4.33                 h7f8727e_1  
libevent                  2.1.12               h8f2d780_0  
libffi                    3.4.2                h6a678d5_6  
libgcc-devel_linux-64     11.2.0               h1234567_1  
libgcc-ng                 11.2.0               h1234567_1  
libgfortran-ng            11.2.0               h00389a5_1  
libgfortran5              11.2.0               h1234567_1  
libgomp                   11.2.0               h1234567_1  
libllvm10                 10.0.1               hbcb73fb_5  
libnetcdf                 4.6.1                h2053bdc_4  
libnghttp2                1.46.0               hce63b2e_0  
libpng                    1.6.37               hbc83047_0  
libpq                     12.9                 h16c4e8d_3  
libprotobuf               3.20.1               h4ff587b_0  
libsodium                 1.0.18               h7b6447c_0  
libssh2                   1.10.0               h8f2d780_0  
libstdcxx-devel_linux-64  11.2.0               h1234567_1  
libstdcxx-ng              11.2.0               h1234567_1  
libthrift                 0.15.0               hcc01f38_0  
libtiff                   4.4.0                hecacb30_2  
libuuid                   1.41.5               h5eee18b_0  
libwebp                   1.2.4                h11a3e52_0  
libwebp-base              1.2.4                h5eee18b_0  
libxcb                    1.15                 h7f8727e_0  
libxkbcommon              1.0.1                hfa300c1_0  
libxml2                   2.9.14               h74e7548_0  
libxslt                   1.1.35               h4e12654_0  
logical-unification       0.4.5              pyhd8ed1ab_0    conda-forge
lxml                      4.9.1           py310h1edc446_0  
lz4-c                     1.9.4                h6a678d5_0  
markupsafe                2.1.1           py310h7f8727e_0  
matplotlib                3.6.2           py310h06a4308_0  
matplotlib-base           3.6.2           py310h945d387_0  
matplotlib-inline         0.1.6           py310h06a4308_0  
minikanren                1.0.3              pyhd8ed1ab_0    conda-forge
mistune                   0.8.4           py310h7f8727e_1000  
mkl                       2021.4.0           h06a4308_640  
mkl-service               2.4.0           py310h7f8727e_0  
mkl_fft                   1.3.1           py310hd6ae3a3_0  
mkl_random                1.2.2           py310h00e6091_0  
multipledispatch          0.6.0           py310h06a4308_0  
munkres                   1.1.4                      py_0  
nbclassic                 0.4.8           py310h06a4308_0  
nbclient                  0.5.13          py310h06a4308_0  
nbconvert                 6.5.4           py310h06a4308_0  
nbformat                  5.7.0           py310h06a4308_0  
ncurses                   6.3                  h5eee18b_3  
nest-asyncio              1.5.6           py310h06a4308_0  
netcdf4                   1.5.7           py310hf533683_0  
notebook                  6.5.2           py310h06a4308_0  
notebook-shim             0.2.2           py310h06a4308_0  
nspr                      4.33                 h295c915_0  
nss                       3.74                 h0370c37_0  
numexpr                   2.8.4           py310h8879344_0  
numpy                     1.23.5          py310hd5efca6_0  
numpy-base                1.23.5          py310h8e6c178_0  
openssl                   1.1.1s               h7f8727e_0  
orc                       1.7.4                h07ed6aa_0  
packaging                 22.0            py310h06a4308_0  
pandas                    1.5.2           py310h1128e8f_0  
pandocfilters             1.5.0              pyhd3eb1b0_0  
parso                     0.8.3              pyhd3eb1b0_0  
pathlib                   1.0.1              pyhd3eb1b0_1  
patsy                     0.5.2           py310h06a4308_1  
pcre                      8.45                 h295c915_0  
pexpect                   4.8.0              pyhd3eb1b0_3  
pickleshare               0.7.5           pyhd3eb1b0_1003  
pillow                    9.3.0           py310hace64e9_1  
pip                       22.3.1          py310h06a4308_0  
platformdirs              2.5.2           py310h06a4308_0  
pluggy                    1.0.0           py310h06a4308_1  
ply                       3.11            py310h06a4308_0  
prometheus_client         0.14.1          py310h06a4308_0  
prompt-toolkit            3.0.36          py310h06a4308_0  
psutil                    5.9.0           py310h5eee18b_0  
ptyprocess                0.7.0              pyhd3eb1b0_2  
pure_eval                 0.2.2              pyhd3eb1b0_0  
pyarrow                   8.0.0           py310h468efa6_0  
pycosat                   0.6.4           py310h5eee18b_0  
pycparser                 2.21               pyhd3eb1b0_0  
pygments                  2.11.2             pyhd3eb1b0_0  
pymc                      5.0.1                hd8ed1ab_0    conda-forge
pymc-base                 5.0.1              pyhd8ed1ab_0    conda-forge
pyopenssl                 22.0.0             pyhd3eb1b0_0  
pyparsing                 3.0.9           py310h06a4308_0  
pyqt                      5.15.7          py310h6a678d5_1  
pyqt5-sip                 12.11.0                  pypi_0    pypi
pyrsistent                0.18.0          py310h7f8727e_0  
pysocks                   1.7.1           py310h06a4308_0  
pytensor                  2.8.11          py310hd685a3f_1    conda-forge
pytensor-base             2.8.11          py310hff52083_1    conda-forge
python                    3.10.8               h7a1cb2a_1  
python-dateutil           2.8.2              pyhd3eb1b0_0  
python-fastjsonschema     2.16.2          py310h06a4308_0  
python_abi                3.10                    2_cp310    conda-forge
pytz                      2022.7          py310h06a4308_0  
pyyaml                    6.0             py310h5eee18b_1  
pyzmq                     23.2.0          py310h6a678d5_0  
qt-main                   5.15.2               h327a75a_7  
qt-webengine              5.15.9               hd2b0992_4  
qtwebkit                  5.212                h4eab89a_4  
re2                       2022.04.01           h295c915_0  
readline                  8.2                  h5eee18b_0  
requests                  2.28.1          py310h06a4308_0  
ruamel.yaml               0.17.21         py310h5eee18b_0  
ruamel.yaml.clib          0.2.6           py310h5eee18b_1  
scikit-learn              1.1.3           py310h6a678d5_0  
scipy                     1.9.3           py310hd5efca6_0  
seaborn                   0.12.1          py310h06a4308_0  
send2trash                1.8.0              pyhd3eb1b0_1  
setuptools                65.5.0          py310h06a4308_0  
sip                       6.6.2           py310h6a678d5_0  
six                       1.16.0             pyhd3eb1b0_1  
snappy                    1.1.9                h295c915_0  
sniffio                   1.2.0           py310h06a4308_1  
soupsieve                 2.3.2.post1     py310h06a4308_0  
sqlite                    3.40.0               h5082296_0  
stack_data                0.2.0              pyhd3eb1b0_0  
statsmodels               0.13.2          py310h7f8727e_0  
sysroot_linux-64          2.12                he073ed8_15    conda-forge
terminado                 0.17.1          py310h06a4308_0  
threadpoolctl             2.2.0              pyh0d69192_0  
tinycss2                  1.2.1           py310h06a4308_0  
tk                        8.6.12               h1ccaba5_0  
toml                      0.10.2             pyhd3eb1b0_0  
tomli                     2.0.1           py310h06a4308_0  
toolz                     0.12.0          py310h06a4308_0  
tornado                   6.2             py310h5eee18b_0  
tqdm                      4.64.1          py310h06a4308_0  
traitlets                 5.7.1           py310h06a4308_0  
typing-extensions         4.4.0           py310h06a4308_0  
typing_extensions         4.4.0           py310h06a4308_0  
tzdata                    2022g                h04d1e81_0  
urllib3                   1.26.13         py310h06a4308_0  
utf8proc                  2.6.1                h27cfd23_0  
wcwidth                   0.2.5              pyhd3eb1b0_0  
webencodings              0.5.1           py310h06a4308_1  
websocket-client          0.58.0          py310h06a4308_4  
wheel                     0.37.1             pyhd3eb1b0_0  
xarray                    2022.11.0       py310h06a4308_0  
xarray-einstats           0.4.0              pyhd8ed1ab_0    conda-forge
xlrd                      2.0.1              pyhd3eb1b0_0  
xlwt                      1.3.0           py310h06a4308_0  
xz                        5.2.8                h5eee18b_0  
yaml                      0.2.5                h7b6447c_0  
zeromq                    4.3.4                h2531618_0  
zlib                      1.2.13               h5eee18b_0  
zstd                      1.5.2                ha4553b6_0
12

Maybe @ricardoV94 would have some guess about what’s happening here?

13

I had a similar problem on OS/X which was caused by PyMC using Anaconda’s version of clang instead of Xcode’s. I updated all my build tools and changed my path to put /usr/bin before Anaconda, which solved the error.

4 Likes
14

+1 here, just updated my machine from Catalina to Ventura 13.3.1 (a) (22E772610a), still trying to use everything as Intel x86 versions via Rosetta2 so I’ll go round a quick loop of reinstalling Xcode etc and see how far I get

15

Returning to this after a little while, and it’s still a world of pain despite several attempts at nuking and rebuilding my system & workflow.

I’d greatly appreciate any pointers! Looks like it’s something to do with the C compiler.

In general I want to develop for cross-compatibilty with Linux machines where my
code will be production-deployed. This means running on Intel x86_64, so what has
worked for me to-date until this latest issue is to set osx-64 (Intel x86_64 via Rosetta2).

Basic machine setup

  • Macbook Air M2 with MacOS Ventura 13.4
  • Xcode Command Line tools installed via xcode-select --install
  • Micromamba installed via Homebrew brew install micromamba
  • Micromamba configured to get x86 packages
$> cat ~/.mambarc
platform: osx-64
channels:
  - conda-forge
  - nodefaults
show_channel_urls: True
  • Shell configured to put homebrew on the CPATH and LIBRARY_PATH (not actually sure why this would be helpful
$>cat ~/.zshrc
...
# potential C compiler path fix for issue https://apple.stackexchange.com/a/414625/192847
export CPATH=/opt/homebrew/include
export LIBRARY_PATH=/opt/homebrew/lib
...

Environment Setup

  • My project env is more complicated, but this is a minimal set.
$> CONDA_SUBDIR=osx-64 micromamba env create --file condaenv_oreum_lab.yml

… quick look

$> cat condaenv.yml

name: oreum_lab
channels:
  - conda-forge
  - nodefaults
dependencies:
- ipykernel
- python==3.10.*
- pymc>=5.4.0

Error seen

  • Very simple steps to recreate

[In]

import pymc as pm
print(f"Running on PyMC v{pm.__version__}")

[Out]

WARNING (pytensor.tensor.blas): Using NumPy C-API based implementation for BLAS functions.
Running on PyMC v5.4.0

… and then

[In]

with pm.Model() as model:
    mu = pm.Normal('mu', mu=0, sigma=1)
    s = pm.Exponential('sigma', lam=1)
    y = pm.Normal('y', mu=mu, sigma=s, observed=[9, 10, 11])

[Out]

ERROR (pytensor.graph.rewriting.basic): Rewrite failure due to: constant_folding
ERROR (pytensor.graph.rewriting.basic): node: InplaceDimShuffle{}(TensorConstant{(1,) of 3})
ERROR (pytensor.graph.rewriting.basic): TRACEBACK:
ERROR (pytensor.graph.rewriting.basic): Traceback (most recent call last):
  File "/Users/jon/micromamba/envs/oreum_lab/lib/python3.10/site-packages/pytensor/graph/rewriting/basic.py", line 1918, in process_node
    replacements = node_rewriter.transform(fgraph, node)
  File "/Users/jon/micromamba/envs/oreum_lab/lib/python3.10/site-packages/pytensor/graph/rewriting/basic.py", line 1078, in transform
    return self.fn(fgraph, node)
  File "/Users/jon/micromamba/envs/oreum_lab/lib/python3.10/site-packages/pytensor/tensor/rewriting/basic.py", line 1138, in constant_folding
    thunk = node.op.make_thunk(node, storage_map, compute_map, no_recycling=[])
  File "/Users/jon/micromamba/envs/oreum_lab/lib/python3.10/site-packages/pytensor/link/c/op.py", line 131, in make_thunk
    return self.make_c_thunk(node, storage_map, compute_map, no_recycling)
  File "/Users/jon/micromamba/envs/oreum_lab/lib/python3.10/site-packages/pytensor/link/c/op.py", line 96, in make_c_thunk
    outputs = cl.make_thunk(
  File "/Users/jon/micromamba/envs/oreum_lab/lib/python3.10/site-packages/pytensor/link/c/basic.py", line 1200, in make_thunk
    cthunk, module, in_storage, out_storage, error_storage = self.__compile__(
  File "/Users/jon/micromamba/envs/oreum_lab/lib/python3.10/site-packages/pytensor/link/c/basic.py", line 1120, in __compile__
    thunk, module = self.cthunk_factory(
  File "/Users/jon/micromamba/envs/oreum_lab/lib/python3.10/site-packages/pytensor/link/c/basic.py", line 1644, in cthunk_factory
    module = cache.module_from_key(key=key, lnk=self)
  File "/Users/jon/micromamba/envs/oreum_lab/lib/python3.10/site-packages/pytensor/link/c/cmodule.py", line 1240, in module_from_key
    module = lnk.compile_cmodule(location)
  File "/Users/jon/micromamba/envs/oreum_lab/lib/python3.10/site-packages/pytensor/link/c/basic.py", line 1543, in compile_cmodule
    module = c_compiler.compile_str(
  File "/Users/jon/micromamba/envs/oreum_lab/lib/python3.10/site-packages/pytensor/link/c/cmodule.py", line 2649, in compile_str
...
ld: library not found for -lSystem
clang-15: error: linker command failed with exit code 1 (use -v to see invocation)

WIP exploring the C installation(s) on my machine since that seems likely to be at fault

Maybe the clang installed by micromamba via my request for osx-64 doesn’t have lSystem?

ld: library not found for -lSystem
clang-15: error: linker command failed with exit code 1 (use -v to see invocation)

Yeah, there’s definitely some monkey business here… I’ll document this as much as I can as I go

1. Xcode installed an Apple Silicon clang, fair enough…

Xcode Command Line Tools appears to have installed arm-64 versions and this is the clang on my (not-yet conda “activated” $PATH). This seems unlikely to be what I want for this project where I want Intel x86_64 aka “amd-64” sometimes aka osx-64 (so many acronyms, it’ll be glorious if/when we can get over this transitional period)

$> which clang
/usr/bin/clang

$> clang -v
Apple clang version 14.0.3 (clang-1403.0.22.14.1)
Target: arm64-apple-darwin22.5.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

This is symlinked to gcc too

2. The local environment clang is x86_64, which I do want

When I do a micromamba activate oreum_lab to configure (rather a lot of) new environment variables, I get this:

$> micromamba activate oreum_lab
$> which clang
/Users/jon/micromamba/envs/oreum_lab/bin/clang

$> clang -v
clang version 15.0.7
Target: x86_64-apple-darwin22.5.0
Thread model: posix
InstalledDir: /Users/jon/micromamba/envs/oreum_lab/bin

3. BUT (aha?) the local environment gcc is still Apple Silicon

Maybe this is the smoking gun… does Pytensor request to use gcc rather than clang?

$> which gcc
/usr/bin/gcc

$> gcc -v
Apple clang version 14.0.3 (clang-1403.0.22.14.1)
Target: arm64-apple-darwin22.5.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

4. Just to confirm what clang is installed into the mamba env

$> micromamba list | grep clang
  clang                          15.0.7        h694c41f_2               conda-forge
  clang-15                       15.0.7        default_hdb78580_2       conda-forge
  clang_osx-64                   15.0.7        h03d6864_3               conda-forge
  clangxx                        15.0.7        default_hdb78580_2       conda-forge
  clangxx_osx-64                 15.0.7        h2133e9c_3               conda-forge
  libclang-cpp15                 15.0.7        default_hdb78580_2       conda-forge

… WIP …
… WIP …
… WIP …

More notes

a. The local python is python3.10 on Intel (via Rosetta2), which I want

As reported by Activity Monitor which shows the process “Kind” is “Intel” rather than “Apple” (Silicon)

b. Current state of $PATH (post activate environment variable setting)

$> echo $PATH | sed -e 's/:/:\n/g'

/Users/jon/micromamba/envs/oreum_lab/bin:
/opt/homebrew/opt/openjdk/bin:
/Users/jon/micromamba/condabin:
/opt/homebrew/opt/openjdk/bin:
/opt/homebrew/bin:
/opt/homebrew/sbin:
/usr/local/bin:
/System/Cryptexes/App/usr/bin:
/usr/bin:
/bin:
/usr/sbin:
/sbin:
...
...

c. Now I know more about clang / gcc etc at least…

16

CC @maresb

17

I’ve added few more clues / diagnoses above… still puzzled tbh. The current major symptom:

ld: library not found for -lSystem
clang-15: error: linker command failed with exit code 1 (use -v to see invocation)

… suggests to me that the clang sourced from conda-forge doesn’t include -lSystem, which seems unlikely.

In any case, I’m going to regress slightly to use mambaforge rather than micromamba, which I foolishly thought would be useful but turns out it fails to compile MKL versions of numpy etc even when explicitly requested. Sigh,

I’m still all ears if anyone has any idea about the clang issue!

This could be something to do with it… Environment variables set wrongly by clang activate scripts · Issue #93 · conda-forge/clang-compiler-activation-feedstock · GitHub

18

The plot thickens…

I don’t see this issue when running the code in ipython (hurray?). Instead this seems to be something to do with if/how/not VSCode is “activating” the mamba environment for the Notebook. Baffling.

Will report back when I regress to the olde way with mamba forge and see what happens

19

Well… it works again. Hurray, but I’m not at all sure what’s different!

I think my entire issue was something caused by VSCode Jupyter extension not properly starting the Kernel for my notebook, and somehow misdirecting C calls… maybe. Possibly due to my using micromamba rather than the more conventional mamba (conda)

This is what finally worked for me, nothing else is really any different. I can still run the pymc commands in ipython and now I can also run them in a Notebook in VSCode.

Excerpt from ~/.mambarc

platform: osx-64
channels:
  - conda-forge
  - nodefaults
show_channel_urls: true

Excerpt from project/Makefile

MAMBADL = https://github.com/conda-forge/miniforge/releases/latest/download/
MAMBAV = Mambaforge-MacOSX-x86_64.sh

mamba:  ## get mamba via mambaforge for MacOS x86_64 (Intel via Rosetta2)
	wget $(MAMBADL)$(MAMBAV) -O $$HOME/mambaforge.sh
	bash $$HOME/mambaforge.sh -b -p $$HOME/.mamba
	export PATH=$$HOME/.mamba/bin:$$PATH; \
		conda init zsh;
	rm $$HOME/mambaforge.sh

create-env: ## create mamba (conda) environment
	export PATH=$$HOME/.mamba/bin:$$PATH; \
		if which mamba; then echo "mamba ready"; else make mamba; fi
	mamba env create --file condaenv_oreum_lab.yml; \
2 Likes