PyMC V4 on M1 MacBook Pro getting this message - WARNING (aesara.tensor.blas): Using NumPy C-API based implementation for BLAS functions.
Install is from Anaconda, just as described in the install instructions.
Any help would be sincerely appreciated.
Hi, it may help to address the blas warnings by using mamba to install pymc v4 as follow:
conda create --name pymc4 -c conda-forge python=3.9
conda activate pymc4
conda install -c conda-forge mamba
mamba install -c conda-forge "pymc>=4.0.0"
Still seeing "WARNING (aesara.tensor.blas): Using NumPy C-API based implementation for BLAS functionsā after install with mamba. Thanks for the recommendation, but it didnāt fix it. Other ideas?
Can you share the output of conda list?
Here is conda list:
(pymc4) ā ~ conda list
aeppl 0.0.31 pyhd8ed1ab_0 conda-forge
aesara 2.6.6 py39h7086893_0 conda-forge
arviz 0.12.1 pyhd8ed1ab_0 conda-forge
brotli 1.0.9 h5eb16cf_7 conda-forge
brotli-bin 1.0.9 h5eb16cf_7 conda-forge
brotlipy 0.7.0 py39h63b48b0_1004 conda-forge
bzip2 1.0.8 h0d85af4_4 conda-forge
c-ares 1.18.1 h0d85af4_0 conda-forge
c-compiler 1.4.2 had99412_0 conda-forge
ca-certificates 2022.5.18.1 h033912b_0 conda-forge
cachetools 5.0.0 pyhd8ed1ab_0 conda-forge
cctools 973.0.1 h351d84c_10 conda-forge
cctools_osx-64 973.0.1 h3eff9a4_10 conda-forge
certifi 2022.5.18.1 py39h6e9494a_0 conda-forge
cffi 1.15.0 py39he338e87_0 conda-forge
cftime 1.6.0 py39h86b5767_1 conda-forge
charset-normalizer 2.0.12 pyhd8ed1ab_0 conda-forge
clang 13.0.1 h694c41f_0 conda-forge
clang-13 13.0.1 default_he082bbe_0 conda-forge
clang_osx-64 13.0.1 h71a8856_2 conda-forge
clangxx 13.0.1 default_he082bbe_0 conda-forge
clangxx_osx-64 13.0.1 heae0f87_2 conda-forge
cloudpickle 2.1.0 pyhd8ed1ab_0 conda-forge
colorama 0.4.4 pyh9f0ad1d_0 conda-forge
compiler-rt 13.0.1 he01351e_0 conda-forge
compiler-rt_osx-64 13.0.1 hd3f61c9_0 conda-forge
conda 4.13.0 py39h6e9494a_1 conda-forge
conda-package-handling 1.8.1 py39h63b48b0_1 conda-forge
cons 0.4.5 pyhd8ed1ab_0 conda-forge
cryptography 37.0.2 py39h9c2a9ce_0 conda-forge
curl 7.83.1 h372c54d_0 conda-forge
cxx-compiler 1.4.2 h1b54a9f_0 conda-forge
cycler 0.11.0 pyhd8ed1ab_0 conda-forge
etuples 0.3.5 pyhd8ed1ab_0 conda-forge
fastprogress 1.0.2 pyhd8ed1ab_0 conda-forge
filelock 3.7.1 pyhd8ed1ab_0 conda-forge
fonttools 4.33.3 py39h701faf5_0 conda-forge
freetype 2.10.4 h4cff582_1 conda-forge
giflib 5.2.1 hbcb3906_2 conda-forge
hdf4 4.2.15 hefd3b78_3 conda-forge
hdf5 1.12.1 nompi_ha60fbc9_104 conda-forge
icu 70.1 h96cf925_0 conda-forge
idna 3.3 pyhd8ed1ab_0 conda-forge
importlib-metadata 4.11.4 py39h6e9494a_0 conda-forge
importlib_metadata 4.11.4 hd8ed1ab_0 conda-forge
jpeg 9e h5eb16cf_1 conda-forge
kiwisolver 1.4.2 py39h7248d28_1 conda-forge
krb5 1.19.3 hb49756b_0 conda-forge
lcms2 2.12 h577c468_0 conda-forge
ld64 609 ha328185_10 conda-forge
ld64_osx-64 609 h6fbe7a8_10 conda-forge
lerc 3.0 he49afe7_0 conda-forge
libarchive 3.5.2 hde4784d_2 conda-forge
libblas 3.9.0 15_osx64_openblas conda-forge
libbrotlicommon 1.0.9 h5eb16cf_7 conda-forge
libbrotlidec 1.0.9 h5eb16cf_7 conda-forge
libbrotlienc 1.0.9 h5eb16cf_7 conda-forge
libcblas 3.9.0 15_osx64_openblas conda-forge
libclang-cpp13 13.0.1 default_he082bbe_0 conda-forge
libcurl 7.83.1 h372c54d_0 conda-forge
libcxx 14.0.5 hc203e6f_0 conda-forge
libdeflate 1.10 h0d85af4_0 conda-forge
libedit 3.1.20191231 h0678c8f_2 conda-forge
libev 4.33 haf1e3a3_1 conda-forge
libffi 3.4.2 h0d85af4_5 conda-forge
libgfortran 5.0.0 9_3_0_h6c81a4c_23 conda-forge
libgfortran5 9.3.0 h6c81a4c_23 conda-forge
libiconv 1.16 haf1e3a3_0 conda-forge
liblapack 3.9.0 15_osx64_openblas conda-forge
libllvm13 13.0.1 h64f94b2_2 conda-forge
libmamba 0.24.0 h2d3d89a_1 conda-forge
libmambapy 0.24.0 py39h3f08081_1 conda-forge
libnetcdf 4.8.1 nompi_h6609ca0_102 conda-forge
libnghttp2 1.47.0 h942079c_0 conda-forge
libopenblas 0.3.20 openmp_hb3cd9ec_0 conda-forge
libpng 1.6.37 h7cec526_2 conda-forge
libsolv 0.7.22 hd9580d2_0 conda-forge
libssh2 1.10.0 h52ee1ee_2 conda-forge
libtiff 4.4.0 hfca7e8f_0 conda-forge
libwebp 1.2.2 h28dabe5_0 conda-forge
libwebp-base 1.2.2 h0d85af4_1 conda-forge
libxcb 1.13 h0d85af4_1004 conda-forge
libxml2 2.9.14 h08a9926_0 conda-forge
libzip 1.8.0 h8b0c345_1 conda-forge
libzlib 1.2.12 h6c3fc93_0 conda-forge
llvm-openmp 14.0.4 ha654fa7_0 conda-forge
llvm-tools 13.0.1 h64f94b2_2 conda-forge
logical-unification 0.4.5 pyhd8ed1ab_0 conda-forge
lz4-c 1.9.3 he49afe7_1 conda-forge
lzo 2.10 haf1e3a3_1000 conda-forge
mamba 0.24.0 py39ha435c47_1 conda-forge
matplotlib-base 3.5.2 py39h64a0072_0 conda-forge
minikanren 1.0.3 pyhd8ed1ab_0 conda-forge
mkl 2022.1.0 h860c996_928 conda-forge
mkl-service 2.4.0 py39h05e0b47_0 conda-forge
multipledispatch 0.6.0 py_0 conda-forge
munkres 1.1.4 pyh9f0ad1d_0 conda-forge
ncurses 6.3 h96cf925_1 conda-forge
netcdf4 1.5.8 nompi_py39he7d1c46_101 conda-forge
numpy 1.22.4 py39h677350a_0 conda-forge
openjpeg 2.4.0 h6e7aa92_1 conda-forge
openssl 1.1.1o hfe4f2af_0 conda-forge
packaging 21.3 pyhd8ed1ab_0 conda-forge
pandas 1.4.2 py39hf72b562_2 conda-forge
pillow 9.1.1 py39h579eac4_1 conda-forge
pip 22.1.2 pyhd8ed1ab_0 conda-forge
pthread-stubs 0.4 hc929b4f_1001 conda-forge
pybind11-abi 4 hd8ed1ab_3 conda-forge
pycosat 0.6.3 py39h63b48b0_1010 conda-forge
pycparser 2.21 pyhd8ed1ab_0 conda-forge
pymc 4.0.0 py39h360fd21_1 conda-forge
pymc-base 4.0.0 py39h6e9494a_1 conda-forge
pyopenssl 22.0.0 pyhd8ed1ab_0 conda-forge
pyparsing 3.0.9 pyhd8ed1ab_0 conda-forge
pysocks 1.7.1 py39h6e9494a_5 conda-forge
python 3.9.13 h57e37ff_0_cpython conda-forge
python-dateutil 2.8.2 pyhd8ed1ab_0 conda-forge
python_abi 3.9 2_cp39 conda-forge
pytz 2022.1 pyhd8ed1ab_0 conda-forge
readline 8.1.2 h3899abd_0 conda-forge
reproc 14.2.3 h0d85af4_0 conda-forge
reproc-cpp 14.2.3 he49afe7_0 conda-forge
requests 2.28.0 pyhd8ed1ab_0 conda-forge
ruamel_yaml 0.15.80 py39h701faf5_1007 conda-forge
scipy 1.8.1 py39hfa1a3ab_0 conda-forge
setuptools 62.3.4 py39h6e9494a_0 conda-forge
sigtool 0.1.3 h57ddcff_0 conda-forge
six 1.16.0 pyh6c4a22f_0 conda-forge
sqlite 3.38.5 hd9f0692_0 conda-forge
tapi 1100.0.11 h9ce4665_0 conda-forge
tbb 2021.5.0 hbb4e6a2_1 conda-forge
tk 8.6.12 h5dbffcc_0 conda-forge
toolz 0.11.2 pyhd8ed1ab_0 conda-forge
tqdm 4.64.0 pyhd8ed1ab_0 conda-forge
typing-extensions 3.10.0.2 hd8ed1ab_0 conda-forge
typing_extensions 3.10.0.2 pyha770c72_0 conda-forge
tzdata 2022a h191b570_0 conda-forge
unicodedata2 14.0.0 py39h63b48b0_1 conda-forge
urllib3 1.26.9 pyhd8ed1ab_0 conda-forge
wheel 0.37.1 pyhd8ed1ab_0 conda-forge
xarray 2022.3.0 pyhd8ed1ab_0 conda-forge
xarray-einstats 0.2.2 pyhd8ed1ab_0 conda-forge
xorg-libxau 1.0.9 h35c211d_0 conda-forge
xorg-libxdmcp 1.1.3 h35c211d_0 conda-forge
xz 5.2.5 haf1e3a3_1 conda-forge
yaml 0.2.5 h0d85af4_2 conda-forge
yaml-cpp 0.7.0 hb486fe8_1 conda-forge
zipp 3.8.0 pyhd8ed1ab_0 conda-forge
zlib 1.2.12 h6c3fc93_0 conda-forge
zstd 1.5.2 ha9df2e0_1 conda-forge
That all looks correct. Whatās the output of:
python -m aesara.misc.check_blas
This machine is only a few months old, and when I got it, I did the āmigrationā process from my previous 2012 MacBook. Iām wondering if something was copied or set up wrong from the old machine? Thanks.
Here is the output of the python -m apsara.misc.check_blas:
(pymc4) ā ~ python -m aesara.misc.check_blas
WARNING (aesara.tensor.blas): Using NumPy C-API based implementation for BLAS functions.
Some results that you can compare against. They were 10 executions
of gemm in float64 with matrices of shape 2000x2000 (M=N=K=2000).
All memory layout was in C order.
CPU tested: Xeon E5345(2.33Ghz, 8M L2 cache, 1333Mhz FSB),
Xeon E5430(2.66Ghz, 12M L2 cache, 1333Mhz FSB),
Xeon E5450(3Ghz, 12M L2 cache, 1333Mhz FSB),
Xeon X5560(2.8Ghz, 12M L2 cache, hyper-threads?)
Core 2 E8500, Core i7 930(2.8Ghz, hyper-threads enabled),
Core i7 950(3.07GHz, hyper-threads enabled)
Xeon X5550(2.67GHz, 8M l2 cache?, hyper-threads enabled)
Libraries tested:
* numpy with ATLAS from distribution (FC9) package (1 thread)
* manually compiled numpy and ATLAS with 2 threads
* goto 1.26 with 1, 2, 4 and 8 threads
* goto2 1.13 compiled with multiple threads enabled
Xeon Xeon Xeon Core2 i7 i7 Xeon Xeon
lib/nb threads E5345 E5430 E5450 E8500 930 950 X5560 X5550
numpy 1.3.0 blas 775.92s
numpy_FC9_atlas/1 39.2s 35.0s 30.7s 29.6s 21.5s 19.60s
goto/1 18.7s 16.1s 14.2s 13.7s 16.1s 14.67s
numpy_MAN_atlas/2 12.0s 11.6s 10.2s 9.2s 9.0s
goto/2 9.5s 8.1s 7.1s 7.3s 8.1s 7.4s
goto/4 4.9s 4.4s 3.7s - 4.1s 3.8s
goto/8 2.7s 2.4s 2.0s - 4.1s 3.8s
openblas/1 14.04s
openblas/2 7.16s
openblas/4 3.71s
openblas/8 3.70s
mkl 11.0.083/1 7.97s
mkl 10.2.2.025/1 13.7s
mkl 10.2.2.025/2 7.6s
mkl 10.2.2.025/4 4.0s
mkl 10.2.2.025/8 2.0s
goto2 1.13/1 14.37s
goto2 1.13/2 7.26s
goto2 1.13/4 3.70s
goto2 1.13/8 1.94s
goto2 1.13/16 3.16s
Test time in float32. There were 10 executions of gemm in
float32 with matrices of shape 5000x5000 (M=N=K=5000)
All memory layout was in C order.
cuda version 8.0 7.5 7.0
gpu
M40 0.45s 0.47s
k80 0.92s 0.96s
K6000/NOECC 0.71s 0.69s
P6000/NOECC 0.25s
Titan X (Pascal) 0.28s
GTX Titan X 0.45s 0.45s 0.47s
GTX Titan Black 0.66s 0.64s 0.64s
GTX 1080 0.35s
GTX 980 Ti 0.41s
GTX 970 0.66s
GTX 680 1.57s
GTX 750 Ti 2.01s 2.01s
GTX 750 2.46s 2.37s
GTX 660 2.32s 2.32s
GTX 580 2.42s
GTX 480 2.87s
TX1 7.6s (float32 storage and computation)
GT 610 33.5s
Some Aesara flags:
blas__ldflags=
compiledir= /Users/shackelford/.aesara/compiledir_macOS-12.4-x86_64-i386-64bit-i386-3.9.13-64
floatX= float64
device= cpu
Some OS information:
sys.platform= darwin
sys.version= 3.9.13 | packaged by conda-forge | (main, May 27 2022, 17:00:52)
[Clang 13.0.1 ]
sys.prefix= /Users/shackelford/opt/anaconda3/envs/pymc4
Some environment variables:
MKL_NUM_THREADS= None
OMP_NUM_THREADS= None
GOTO_NUM_THREADS= None
Numpy config: (used when the Aesara flag āblas__ldflagsā is empty)
blas_info:
libraries = [ācblasā, āblasā, ācblasā, āblasā]
library_dirs = [ā/Users/shackelford/opt/anaconda3/envs/pymc4/libā]
include_dirs = [ā/Users/shackelford/opt/anaconda3/envs/pymc4/includeā]
language = c
define_macros = [(āHAVE_CBLASā, None)]
blas_opt_info:
define_macros = [(āNO_ATLAS_INFOā, 1), (āHAVE_CBLASā, None)]
libraries = [ācblasā, āblasā, ācblasā, āblasā]
library_dirs = [ā/Users/shackelford/opt/anaconda3/envs/pymc4/libā]
include_dirs = [ā/Users/shackelford/opt/anaconda3/envs/pymc4/includeā]
language = c
lapack_info:
libraries = [ālapackā, āblasā, ālapackā, āblasā]
library_dirs = [ā/Users/shackelford/opt/anaconda3/envs/pymc4/libā]
language = f77
lapack_opt_info:
libraries = [ālapackā, āblasā, ālapackā, āblasā, ācblasā, āblasā, ācblasā, āblasā]
library_dirs = [ā/Users/shackelford/opt/anaconda3/envs/pymc4/libā]
language = c
define_macros = [(āNO_ATLAS_INFOā, 1), (āHAVE_CBLASā, None)]
include_dirs = [ā/Users/shackelford/opt/anaconda3/envs/pymc4/includeā]
Supported SIMD extensions in this NumPy install:
baseline = SSE,SSE2,SSE3
found = SSSE3,SSE41,POPCNT,SSE42
not found = AVX,F16C,FMA3,AVX2,AVX512F,AVX512CD,AVX512_KNL,AVX512_SKX,AVX512_CLX,AVX512_CNL,AVX512_ICL
Numpy dot module: numpy
Numpy location: /Users/shackelford/opt/anaconda3/envs/pymc4/lib/python3.9/site-packages/numpy/init.py
Numpy version: 1.22.4
We executed 10 calls to gemm with a and b matrices of shapes (5000, 5000) and (5000, 5000).
Total execution time: 23.88s on CPU (with direct Aesara binding to blas).
Try to run this script a few times. Experience shows that the first time is not as fast as following calls. The difference is not big, but consistent.
Oh you have an intel chip. In that case you need to install mkl-service as well.
Wow, thatās news to me. My computer thinks different. See below. Not sure what to do next. I guess major reset to factory setting. Thanks
Hardware Overview:
Model Name: MacBook Pro
Model Identifier: MacBookPro18,3
Chip: Apple M1 Pro
Total Number of Cores: 8 (6 performance and 2 efficiency)
Memory: 16 GB
System Firmware Version: 7459.121.3
OS Loader Version: 7459.121.3
Serial Number (system): HGXQC7DQF7
Hardware UUID: 4EA2ADE3-6B7F-53B7-BAF6-EDBF59A7D3B3
Provisioning UDID: 00006000-000411D63E62801E
Activation Lock Status: Disabled
Can you describe how you migrated from your prior machine? That might explain some of the confusion.
Oh sorry, I was confused, you do have an M1 chip. Then I donāt know why youāre seeing this.
Are you running in Rosetta compatibility mode by any chance?
Possible solution described here:
In short, you might need to install/re-install xcode-select via the following terminal command:
xcode-select --install
I did a complete factory wipe/reset because I figured that my problem was due to some unknown issue with the original install āmigrationā from my old MacBook. I installed Xcode, Anaconda3, then created pymc_dev environment according to install instructions. Then, python, >>> import pymc. I then got the same: WARNING (Using NumPy C-APIā¦).
BTW, when I ran the Anaconda installer, I got the message: To open you need to install Rosetta. Do you want to install it now?
I said āinstallā.
Not sure what to try next. Any further input or suggestions would be appreciated. Thanks
You are almost certainly running Python via Rosetta - Rosetta is an Apple emulation software that letās you take software that is compiled for intel CPUs and run it on the new M1 CPUs. Iām pretty sure that youāve somehow installed Anaconda in a way that told your Mac to use Rosetta for installing it.
I suggest you:
Uninstall anaconda completely
Install either anaconda or miniconda (direct miniconda link for the M1 version here)
Optionally conda install mamba (highly recommend, and then use mamba instead of conda)
Install pymc according to install instructions.
Hi Thomas, thanks for your suggestion. I followed your directions and used the M1 miniconda. I installed mamba, created a new environment (pymc4), and then used mamba to install pymc. I got some new errors this time, plus the warning. See result:
(pymc4) ā ~ python
Python 3.9.13 | packaged by conda-forge | (main, May 27 2022, 17:01:00)
[Clang 13.0.1 ] on darwin
Type āhelpā, ācopyrightā, ācreditsā or ālicenseā for more information.
import pymc
ld: library not found for -lSystem
clang-13: error: linker command failed with exit code 1 (use -v to see invocation)
WARNING (aesara.tensor.blas): Using NumPy C-API based implementation for BLAS functions.
Fixed! PyMC is now working without Warnings or other issues! After having the last error, I went to Xcode and installed an update. Then went back to python and everything worked without error!
Thanks to everybody that helped me get this worked out. Your help is greatly appreciated.
Mark