Is there a workaround using pip ? I personally do not like using conda (slow, overblown) and pip just works fine for most things. I get the warning with pip (and conda regardless, for that matter). I wouldn’t mind spending time specifying the correct paths for BLAS … Any hint? (I’m on linux, Ubuntu specifically)
1 Like
Getting the warning on conda is strange. Regarding overblown, there are alternatives, mamba is much faster than conda and micromamba is smaller (and still faster). Are you installing from conda-forge channel? Sometimes defaults have to be removed explicitly from the available channels.
Anyway I know that’s not what you were asking for 
Thanks @ricardoV94 for the micromamba suggestion (I had used mamba in the past after conda-related frustration grew to an unsustainable level). I can confirm creating the local environment with micromamba does not trigger the BLAS warning, whereas it does with pip. However I can’t help but lamenting the order of magnitude longer setup time by all conda-like tools compared to native pip, and very much wish pip documentation would exist as well.
1 Like
pip is mostly insufficient because of the C-backend and linking to libraries like BLAS. The good news is we are actually planning to drop this backend or make it non-default in the mid-near future in favor of the numba backend. If/when that happens pip install should again suffice.
For the time being you can open a request issue on the PyTensor repo asking for instructions. Do double check if somebody already opened an issue or discussion, as I remember something similar.
2 Likes
I there I recently started getting this warning. I have been using pymc for months and not seen it. I don’t think I have changed anything in my pymc environment. However, I am on windows 10 and I noticed that something in that the windows virus checker is blocking something in pymc
import pymc as pm
print(f"Running on PyMC v{pm.version}")
yields the following
WARNING (pytensor.tensor.blas): Using NumPy C-API based implementation for BLAS functions.
Running on PyMC v5.6.1
it also triggers the antivirus. I am not an admin on the the machine to see what is being blocked by our IT department. But if you have any suggestions it would be helpful. If I rerun the cell the warnings don’t happen…
What platform are you on and how did you install pymc? Can you provide the output of conda list in the environment you are currently using?
I am using windows 10 (using anaconda and VSCode).
Installed via conda install on the command line
We think it is the m2w64-toolchain triggering the block
conda list output
Name Version Build Channel
arviz 0.17.1 py311haa95532_0
asttokens 2.0.5 pyhd3eb1b0_0
bambi 0.13.0 pyhd8ed1ab_0 conda-forge
blas 1.0 mkl
bottleneck 1.3.7 py311hd7041d2_0
brotli 1.0.9 h2bbff1b_8
brotli-bin 1.0.9 h2bbff1b_8
bzip2 1.0.8 h2bbff1b_6
ca-certificates 2024.3.11 haa95532_0
cachetools 5.3.3 py311haa95532_0
cairo 1.16.0 haedb8bc_5
cloudpickle 2.2.1 py311haa95532_0
colorama 0.4.6 py311haa95532_0
comm 0.2.1 py311haa95532_0
cons 0.4.6 py311haa95532_0
contourpy 1.2.0 py311h59b6b97_0
cycler 0.11.0 pyhd3eb1b0_0
debugpy 1.6.7 py311hd77b12b_0
decorator 5.1.1 pyhd3eb1b0_0
etuples 0.3.9 py311haa95532_0
executing 0.8.3 pyhd3eb1b0_0
expat 2.6.2 h63175ca_0 conda-forge
fastprogress 1.0.0 pyhb85f177_0
filelock 3.13.1 py311haa95532_0
font-ttf-dejavu-sans-mono 2.37 hab24e00_0 conda-forge
font-ttf-inconsolata 3.000 h77eed37_0 conda-forge
font-ttf-source-code-pro 2.038 h77eed37_0 conda-forge
font-ttf-ubuntu 0.83 h77eed37_2 conda-forge
fontconfig 2.14.1 h9c4af85_2
fonts-conda-ecosystem 1 0 conda-forge
fonts-conda-forge 1 0 conda-forge
fonttools 4.51.0 py311h2bbff1b_0
formulae 0.5.3 pyhd8ed1ab_0 conda-forge
freetype 2.12.1 ha860e81_0
fribidi 1.0.10 h8d14728_0 conda-forge
getopt-win32 0.1 hcfcfb64_1 conda-forge
glib 2.78.4 hd77b12b_0
glib-tools 2.78.4 hd77b12b_0
graphite2 1.3.13 h63175ca_1003 conda-forge
graphviz 7.1.0 h51cb2cd_0 conda-forge
gts 0.7.6 h6b5321d_4 conda-forge
h5netcdf 1.2.0 py311haa95532_0
h5py 3.11.0 py311h765d3d0_0
harfbuzz 6.0.0 h196d34a_1 conda-forge
hdf5 1.12.1 h51c971a_3
icc_rt 2022.1.0 h6049295_2
icu 72.1 h63175ca_0 conda-forge
intel-openmp 2023.1.0 h59b6b97_46320
ipykernel 6.28.0 py311haa95532_0
ipython 8.20.0 py311haa95532_0
jedi 0.18.1 py311haa95532_1
jpeg 9e h2bbff1b_1
jupyter_client 8.6.0 py311haa95532_0
jupyter_core 5.5.0 py311haa95532_0
kiwisolver 1.4.4 py311hd77b12b_0
lcms2 2.12 h83e58a3_0
lerc 3.0 hd77b12b_0
libbrotlicommon 1.0.9 h2bbff1b_8
libbrotlidec 1.0.9 h2bbff1b_8
libbrotlienc 1.0.9 h2bbff1b_8
libdeflate 1.17 h2bbff1b_1
libexpat 2.6.2 h63175ca_0 conda-forge
libffi 3.4.4 hd77b12b_1
libgd 2.3.3 ha43c60c_1
libglib 2.78.4 ha17d25a_0
libiconv 1.17 hcfcfb64_2 conda-forge
libpng 1.6.39 h8cc25b3_0
libsodium 1.0.18 h62dcd97_0
libtiff 4.5.1 hd77b12b_0
libwebp-base 1.3.2 h2bbff1b_0
libxml2 2.10.4 h0ad7f3c_2
logical-unification 0.4.6 py311haa95532_0
lz4-c 1.9.4 h2bbff1b_1
m2w64-binutils 2.25.1 5
m2w64-bzip2 1.0.6 6
m2w64-crt-git 5.0.0.4636.2595836 2
m2w64-gcc 5.3.0 6
m2w64-gcc-ada 5.3.0 6
m2w64-gcc-fortran 5.3.0 6
m2w64-gcc-libgfortran 5.3.0 6
m2w64-gcc-libs 5.3.0 7
m2w64-gcc-libs-core 5.3.0 7
m2w64-gcc-objc 5.3.0 6
m2w64-gmp 6.1.0 2
m2w64-headers-git 5.0.0.4636.c0ad18a 2
m2w64-isl 0.16.1 2
m2w64-libiconv 1.14 6
m2w64-libmangle-git 5.0.0.4509.2e5a9a2 2
m2w64-libwinpthread-git 5.0.0.4634.697f757 2
m2w64-make 4.1.2351.a80a8b8 2
m2w64-mpc 1.0.3 3
m2w64-mpfr 3.1.4 4
m2w64-pkg-config 0.29.1 2
m2w64-toolchain 5.3.0 7
m2w64-tools-git 5.0.0.4592.90b8472 2
m2w64-windows-default-manifest 6.4 3
m2w64-winpthreads-git 5.0.0.4634.697f757 2
m2w64-zlib 1.2.8 10
matplotlib-base 3.8.4 py311hf62ec03_0
matplotlib-inline 0.1.6 py311haa95532_0
minikanren 1.0.3 py311haa95532_0
mkl 2023.1.0 h6b88ed4_46358
mkl-service 2.4.0 py311h2bbff1b_1
mkl_fft 1.3.8 py311h2bbff1b_0
mkl_random 1.2.4 py311h59b6b97_0
msys2-conda-epoch 20160418 1
multipledispatch 0.6.0 py311haa95532_0
nest-asyncio 1.6.0 py311haa95532_0
numexpr 2.8.7 py311h1fcbade_0
numpy 1.25.2 py311hdab7c0b_0
numpy-base 1.25.2 py311hd01c5d8_0
openjpeg 2.4.0 h4fc8c34_0
openssl 3.3.0 hcfcfb64_0 conda-forge
packaging 23.2 py311haa95532_0
pandas 2.2.1 py311hea22821_0
pango 1.50.12 hdffb7b3_1 conda-forge
parso 0.8.3 pyhd3eb1b0_0
pcre2 10.42 h0ff8eda_1
pillow 10.3.0 py311h2bbff1b_0
pip 24.0 py311haa95532_0
pixman 0.43.4 h63175ca_0 conda-forge
platformdirs 3.10.0 py311haa95532_0
prompt-toolkit 3.0.43 py311haa95532_0
prompt_toolkit 3.0.43 hd3eb1b0_0
psutil 5.9.0 py311h2bbff1b_0
pure_eval 0.2.2 pyhd3eb1b0_0
pybind11-abi 5 hd3eb1b0_0
pygments 2.15.1 py311haa95532_1
pymc 5.6.1 py311h5cc824b_0
pyparsing 3.0.9 py311haa95532_0
pytensor 2.12.3 py311hf62ec03_0
python 3.11.9 he1021f5_0
python-dateutil 2.9.0post0 py311haa95532_0
python-graphviz 0.20.3 pyh717bed2_0 conda-forge
python-tzdata 2023.3 pyhd3eb1b0_0
pytz 2024.1 py311haa95532_0
pywin32 305 py311h2bbff1b_0
pyzmq 25.1.2 py311hd77b12b_0
scipy 1.12.0 py311h9f229c6_0
setuptools 69.5.1 py311haa95532_0
six 1.16.0 pyhd3eb1b0_1
sqlite 3.45.3 h2bbff1b_0
stack_data 0.2.0 pyhd3eb1b0_0
tbb 2021.8.0 h59b6b97_0
tk 8.6.14 h0416ee5_0
toolz 0.12.0 py311haa95532_0
tornado 6.3.3 py311h2bbff1b_0
traitlets 5.7.1 py311haa95532_0
typing-extensions 4.9.0 py311haa95532_1
typing_extensions 4.9.0 py311haa95532_1
tzdata 2024a h04d1e81_0
ucrt 10.0.22621.0 h57928b3_0 conda-forge
unicodedata2 15.1.0 py311h2bbff1b_0
vc 14.2 h21ff451_1
vc14_runtime 14.38.33130 h82b7239_18 conda-forge
vs2015_runtime 14.38.33130 hcb4865c_18 conda-forge
wcwidth 0.2.5 pyhd3eb1b0_0
wheel 0.43.0 py311haa95532_0
xarray 2023.6.0 py311haa95532_0
xarray-einstats 0.6.0 py311haa95532_0
xz 5.4.6 h8cc25b3_1
zeromq 4.3.5 hd77b12b_0
zlib 1.2.13 h8cc25b3_1
zstd 1.5.5 hd43e919_2
It looks like you didn’t use the installation instructions (found here). Doing so should get you the C tool chain and should also avoid mixing channels (which is generally a bad idea). If you can do so, I would create a new environment following the procedure outlined at that link and see where that gets you.
Hi I can try that,
But what is it in the above that make you say this? I am a bit hazy what I did because it was a about 6 months ago and it has been working until this week. I am reasonably confident this is an internal IT policy change. I thought I did follow those instructions, but do remember getting warning to install the m2w64 toolchain (which I did). I then remember having to down rev numpy.
The mixture of channels is not a good sign (in general) and the fact that PyMC itself was not installed from the conda-forge channel is further evidence that the suggested instructions were not used (note the -c conda-forge in the instructions). The suggested installation instructions preferentially pull from conda-forge and pulling from a single channel minimizes the chances that conda tries to mix versions and dependencies. If your installation was working but you were relying on locally installed C tool chain ingredients (i.e., not installed via conda) and now something outside your conda env has now changes (e.g., via IT policies), then I guess it’s possible your “working environment” could have stopped working (in the sense that it can no longer find the relevant C pieces and falls back on the much, much slower BLAS implementation). Regardless, environments should generally be disposable and, if they aren’t, it’s a sign that something isn’t quite right.
Hi Again thanks for your support grinning:
I just cut and pasted the commands into the command prompt. I tried running my script.
I got a warning in VSCode about no ipykernel (I am using jupyter notebooks). So I installed ipkernel via conda which is probably where the mixed channels came from ( I will try with the conda-forge )
but then I got these warnings
WARNING (pytensor.configdefaults): g++ not available, if using conda: conda install m2w64-toolchain
WARNING (pytensor.configdefaults): g++ not detected! PyTensor will be unable to compile C-implementations and will default to Python. Performance may be severely degraded. To remove this warning, set PyTensor flags cxx to an empty string.
WARNING (pytensor.tensor.blas): Using NumPy C-API based implementation for BLAS functions.
In that case, I will defer to any suggestions that @maresb might have.
Hey, sorry I’m just noticing this thread now.
I’d recommend creating a completely new environment with the command
conda create -n name-of-new-environment -c conda-forge -c nodefaults ipykernel "pymc>=5"
(If the conda command runs slowly, I’d recommend mamba as a drop-in replacement.)
In case you see any more warnings, please attach a new conda list output.
1 Like
I need to use PIP in the current state, so these instructions using non-PIP tools like Conda are not currently helpful.
Is there a github issue or discussion on these planned changes?
You can already use numba backend for many cases, just set the global pytensor.config.mode=”NUMBA” and forbid PyTensor from ever attempting C stuff with pytensor.config.cxx=""
With these configs you can use PyMC from pip install without immediate drawbacks. However numba will sometimes be slower (special compile times), and not all operations are yet supported.
We don’t have an issue for switching the default backend but you can follow, if interested, our attempt run the whole PyTensor suite in the numba backend: Run whole test suite on numba backend by ricardoV94 · Pull Request #811 · pymc-devs/pytensor · GitHub
That will have to work before we switch the default.
2 Likes
Thanks @ricardoV94 , I will try using the NUMBA backend.
Short summary:
It appears when PyMC is installed by itself it installs PyTensor but for some reason does not install gcc and g++, so it defaults to the system compiler. When PyTensor is installed by itself gcc and g++ are installed in the mamba environment as expected. From @cluhmann 's comments it looks like it’s known that some conda installations can pull from the wrong channel and result in not installing the C tool chain, but I think I demonstrate that I’m pulling only from conda-forge.
Should I try the Numba workaround? And if so do I use the Conda version or Pip version of PyMC?
pytensor.config.mode=”NUMBA”
pytensor.config.cxx=""
Also if NUMBA works would models previously generated with default settings be reproducible?
Full post:
I’m having a similar issue with code that previously worked and I really need to be able to reproduce, ideally with the same compiler. I probably messed myself up by installing a new package in my mamba environment, although I also installed an updated system g++ which for whatever reason seems to keep being found as the default compiler, even though it appears g++ is independently installed in the separated mamba environment. As far as I can tell there may be an issue with the conda/mamba package for PyMC or PyTensor so it doesn’t choose the correct g++. Even manually changing the pytensor.config.cxx variable to the /bin/g++ did not seem to fix the issue.
I’ll try to create a minimal reproduction to guarantee that my analysis environment isn’t responsible.
- System g++, no mamba environment
> which g++
/usr/local/bin/g++
g++ --version
g++ (GCC) 15.0.0 20240718 (experimental)
- Mimimal pymc environment, adapted from @maresb 's suggestion to use Mamba and a path prefix.
mamba create --prefix pymc-env -c conda-forge -c nodefaults ipykernel "pymc>=5"
Selected package versions
Blockquote
pymc 5.16.2 hd8ed1ab_0 conda-forge
pymc-base 5.16.2 pyhd8ed1ab_0 conda-forge
pytensor 2.23.0 py312h526ad5a_0
pytensor-base 2.25.3 py312h25a0e75_0 conda-forge
python 3.12.5 h2ad013b_0_cpython conda-forge
Tried to pick out the compilers installed by mamba
Blockquote
_libgcc_mutex 0.1 conda_forge conda-forge
libgcc-ng 14.1.0 h77fa898_0 conda-forge
Verified that system g++ was found in the mamba environment
(pymc-env) >>> which g++
/usr/local/bin/g++
Opened Python prompt and also verified PyTensor was also finding system g++
>>> print(pytensor.config.cxx)
/usr/local/bin/g++
I wanted to see what would happen with a minimal installation of the same version of PyTensor
mamba create --prefix pytensor-env -c conda-forge -c nodefaults ipykernel "pytensor=2.23.0"
Blockquote
pytensor 2.23.0 py312h7070661_0 conda-forge
pytensor-base 2.23.0 py312h266a736_0 conda-forge
It apparently installs gcc and g++
Blockquote
_libgcc_mutex 0.1 conda_forge conda-forge
gcc 12.4.0 h236703b_0 conda-forge
gcc_impl_linux-64 12.4.0 hb2e57f8_0 conda-forge
gcc_linux-64 12.4.0 h6b7512a_0 conda-forge
gxx 12.4.0 h236703b_0 conda-forge
gxx_impl_linux-64 12.4.0 h613a52c_0 conda-forge
gxx_linux-64 12.4.0 h8489865_0 conda-forge
As far as I can tell the only package in common is “libgcc_mutex”. I did my best to only pick out relevant packages but I may have missed some.
Blockquote
_libgcc_mutex 0.1 conda_forge conda-forge
Verified that in the PyTensor only environment the correct g++ is found.
(pytensor-env) >>> which g++
/home/grant/scratch/pytensor-env/bin/g++
g++ (conda-forge gcc 12.4.0-0) 12.4.0
In a Python prompt it did return a warning about BLAS, which was not displayed in the “PyMC” minimal environment.
>>> import pytensor
WARNING (pytensor.tensor.blas): Using NumPy C-API based implementation for BLAS functions.
>>> print(pytensor.config.cxx)
/home/grant/scratch/pytensor-env/bin/g++
2 Likes