Hi @ricardoV94 , I surely wish I could “incredibly” speed my currently 2-3h long calculation (including 30 min pure compile + NUTS init). Unfortunately since I have not been able to do so even from a brand new conda env as in the install doc, I have nothing to miss! On the contrary, on my most recent attempt on a supercomputer, the pymc with conda install actually stalls entirely on pm.sample
, whereas the pip
install runs jusst fine ! I checked with numpy.show_config()
that my pip install defaults to the system’s “openblas” library:
In [2]: np.show_config()
Build Dependencies:
blas:
detection method: pkgconfig
found: true
include directory: /usr/local/include
lib directory: /usr/local/lib
name: openblas64
openblas configuration: USE_64BITINT=1 DYNAMIC_ARCH=1 DYNAMIC_OLDER= NO_CBLAS=
NO_LAPACK= NO_LAPACKE= NO_AFFINITY=1 USE_OPENMP= HASWELL MAX_THREADS=2
pc file directory: /usr/local/lib/pkgconfig
version: 0.3.23.dev
lapack:
detection method: internal
found: true
include directory: unknown
lib directory: unknown
name: dep140213194937296
openblas configuration: unknown
pc file directory: unknown
version: 1.26.4
whereas the stalling conda install indicates:
blas_info:
libraries = ['cblas', 'blas', 'cblas', 'blas']
library_dirs = ['/home/perrette/.conda/envs/pymc-5.9.0/lib']
include_dirs = ['/home/perrette/.conda/envs/pymc-5.9.0/include']
language = c
define_macros = [('HAVE_CBLAS', None)]
blas_opt_info:
define_macros = [('NO_ATLAS_INFO', 1), ('HAVE_CBLAS', None)]
libraries = ['cblas', 'blas', 'cblas', 'blas']
library_dirs = ['/home/perrette/.conda/envs/pymc-5.9.0/lib']
include_dirs = ['/home/perrette/.conda/envs/pymc-5.9.0/include']
language = c
lapack_info:
libraries = ['lapack', 'blas', 'lapack', 'blas']
library_dirs = ['/home/perrette/.conda/envs/pymc-5.9.0/lib']
language = f77
...
And it seems conda installed everything correctly (though possibly redundantly !?):
(.venv-pymc59-py311) [perrette@login04 slr-tidegauges-future]$ ls /home/perrette/.conda/envs/pymc-5.9.0/lib/*lapack* -dl
lrwxrwxrwx 1 perrette users 23 17 set 16.31 /home/perrette/.conda/envs/pymc-5.9.0/lib/liblapacke.so -> libopenblasp-r0.3.27.so
lrwxrwxrwx 1 perrette users 23 17 set 16.31 /home/perrette/.conda/envs/pymc-5.9.0/lib/liblapacke.so.3 -> libopenblasp-r0.3.27.so
lrwxrwxrwx 1 perrette users 23 17 set 16.31 /home/perrette/.conda/envs/pymc-5.9.0/lib/liblapack.so -> libopenblasp-r0.3.27.so
lrwxrwxrwx 1 perrette users 23 17 set 16.31 /home/perrette/.conda/envs/pymc-5.9.0/lib/liblapack.so.3 -> libopenblasp-r0.3.27.so
lrwxrwxrwx 1 perrette users 27 17 set 16.31 /home/perrette/.conda/envs/pymc-5.9.0/lib/libmkl_scalapack_ilp64.so -> libmkl_scalapack_ilp64.so.2
-rwxr-xr-x 5 perrette users 7718768 30 giu 2023 /home/perrette/.conda/envs/pymc-5.9.0/lib/libmkl_scalapack_ilp64.so.2
lrwxrwxrwx 1 perrette users 26 17 set 16.31 /home/perrette/.conda/envs/pymc-5.9.0/lib/libmkl_scalapack_lp64.so -> libmkl_scalapack_lp64.so.2
-rwxr-xr-x 5 perrette users 7728456 30 giu 2023 /home/perrette/.conda/envs/pymc-5.9.0/lib/libmkl_scalapack_lp64.so.2
(.venv-pymc59-py311) [perrette@login04 slr-tidegauges-future]$ ls /home/perrette/.conda/envs/pymc-5.9.0/lib/*blas* -dl
lrwxrwxrwx 1 perrette users 23 17 set 16.31 /home/perrette/.conda/envs/pymc-5.9.0/lib/libblas.so -> libopenblasp-r0.3.27.so
lrwxrwxrwx 1 perrette users 23 17 set 16.31 /home/perrette/.conda/envs/pymc-5.9.0/lib/libblas.so.3 -> libopenblasp-r0.3.27.so
lrwxrwxrwx 1 perrette users 23 17 set 16.31 /home/perrette/.conda/envs/pymc-5.9.0/lib/libcblas.so -> libopenblasp-r0.3.27.so
lrwxrwxrwx 1 perrette users 23 17 set 16.31 /home/perrette/.conda/envs/pymc-5.9.0/lib/libcblas.so.3 -> libopenblasp-r0.3.27.so
lrwxrwxrwx 1 perrette users 22 17 set 16.31 /home/perrette/.conda/envs/pymc-5.9.0/lib/libopenblas.a -> libopenblasp-r0.3.27.a
-rw-r--r-- 4 perrette users 64261142 8 lug 10.12 /home/perrette/.conda/envs/pymc-5.9.0/lib/libopenblasp-r0.3.27.a
-rwxr-xr-x 4 perrette users 36518408 8 lug 10.12 /home/perrette/.conda/envs/pymc-5.9.0/lib/libopenblasp-r0.3.27.so
lrwxrwxrwx 1 perrette users 23 17 set 16.31 /home/perrette/.conda/envs/pymc-5.9.0/lib/libopenblas.so -> libopenblasp-r0.3.27.so
lrwxrwxrwx 1 perrette users 23 17 set 16.31 /home/perrette/.conda/envs/pymc-5.9.0/lib/libopenblas.so.0 -> libopenblasp-r0.3.27.so
I see the thread WARNING (pytensor.tensor.blas) - #38 by GrantDaly has been updated, I’ll take another look. (the pip-friendly NUMBA backend seems promising).