Performance tip if you're on ARM64 (Apple's M1/M2 chips): Install accelerate

absl-py                   1.4.0              pyhd8ed1ab_0    conda-forge
appnope                   0.1.3              pyhd8ed1ab_0    conda-forge
arviz                     0.16.1             pyhd8ed1ab_0    conda-forge
asttokens                 2.4.0              pyhd8ed1ab_0    conda-forge
backcall                  0.2.0              pyh9f0ad1d_0    conda-forge
backports                 1.0                pyhd8ed1ab_3    conda-forge
backports.functools_lru_cache 1.6.5              pyhd8ed1ab_0    conda-forge
blackjax                  0.9.6              pyhd8ed1ab_0    conda-forge
brotli                    1.1.0                h0dc2134_0    conda-forge
brotli-bin                1.1.0                h0dc2134_0    conda-forge
brotli-python             1.1.0           py311hdf8f085_0    conda-forge
bzip2                     1.0.8                h0d85af4_4    conda-forge
c-ares                    1.19.1               h0dc2134_0    conda-forge
ca-certificates           2023.7.22            h8857fd0_0    conda-forge
cached-property           1.5.2                hd8ed1ab_1    conda-forge
cached_property           1.5.2              pyha770c72_1    conda-forge
cachetools                5.3.1              pyhd8ed1ab_0    conda-forge
certifi                   2023.7.22          pyhd8ed1ab_0    conda-forge
charset-normalizer        3.2.0              pyhd8ed1ab_0    conda-forge
cloudpickle               2.2.1              pyhd8ed1ab_0    conda-forge
colorama                  0.4.6              pyhd8ed1ab_0    conda-forge
comm                      0.1.4              pyhd8ed1ab_0    conda-forge
cons                      0.4.6              pyhd8ed1ab_0    conda-forge
contourpy                 1.1.0           py311h5fe6e05_0    conda-forge
cycler                    0.11.0             pyhd8ed1ab_0    conda-forge
debugpy                   1.7.0           py311hdf8f085_0    conda-forge
decorator                 5.1.1              pyhd8ed1ab_0    conda-forge
et_xmlfile                1.1.0           py311hecd8cb5_0  
etuples                   0.3.9              pyhd8ed1ab_0    conda-forge
exceptiongroup            1.1.3              pyhd8ed1ab_0    conda-forge
executing                 1.2.0              pyhd8ed1ab_0    conda-forge
fastprogress              1.0.3              pyhd8ed1ab_0    conda-forge
filelock                  3.12.3             pyhd8ed1ab_0    conda-forge
fonttools                 4.42.1          py311h2725bcf_0    conda-forge
freetype                  2.12.1               h3f81eb7_1    conda-forge
h5netcdf                  1.2.0              pyhd8ed1ab_0    conda-forge
h5py                      3.9.0           nompi_py311hf255e76_102    conda-forge
hdf5                      1.14.2          nompi_hedada53_100    conda-forge
idna                      3.4                pyhd8ed1ab_0    conda-forge
importlib-metadata        6.8.0              pyha770c72_0    conda-forge
importlib_metadata        6.8.0                hd8ed1ab_0    conda-forge
ipykernel                 6.25.2             pyh1050b4e_0    conda-forge
ipython                   8.15.0             pyh31c8845_0    conda-forge
jax                       0.4.14             pyhd8ed1ab_1    conda-forge
jaxlib                    0.4.14          cpu_py311h8b48d28_1    conda-forge
jaxopt                    0.8                pyhd8ed1ab_0    conda-forge
jedi                      0.19.0             pyhd8ed1ab_0    conda-forge
joblib                    1.2.0           py311hecd8cb5_0  
jupyter_client            8.3.1              pyhd8ed1ab_0    conda-forge
jupyter_core              5.3.1           py311h6eed73b_0    conda-forge
kiwisolver                1.4.5           py311h5fe6e05_0    conda-forge
krb5                      1.21.2               hb884880_0    conda-forge
lcms2                     2.15                 h2dcdeff_1    conda-forge
lerc                      4.0.0                hb486fe8_0    conda-forge
libabseil                 20230125.3      cxx17_h000cb23_0    conda-forge
libaec                    1.0.6                hf0c8a7f_1    conda-forge
libblas                   3.9.0           19_osx64_accelerate    conda-forge
libbrotlicommon           1.1.0                h0dc2134_0    conda-forge
libbrotlidec              1.1.0                h0dc2134_0    conda-forge
libbrotlienc              1.1.0                h0dc2134_0    conda-forge
libcblas                  3.9.0           19_osx64_accelerate    conda-forge
libcurl                   8.2.1                h5f667d7_0    conda-forge
libcxx                    16.0.6               hd57cbcb_0    conda-forge
libdeflate                1.18                 hac1461d_0    conda-forge
libedit                   3.1.20191231         h0678c8f_2    conda-forge
libev                     4.33                 haf1e3a3_1    conda-forge
libexpat                  2.5.0                hf0c8a7f_1    conda-forge
libffi                    3.4.2                h0d85af4_5    conda-forge
libgfortran               5.0.0           13_2_0_h97931a8_1    conda-forge
libgfortran5              13.2.0               h2873a65_1    conda-forge
libgrpc                   1.56.2               he6801ca_1    conda-forge
libjpeg-turbo             2.1.5.1              hb7f2c08_0    conda-forge
liblapack                 3.9.0           19_osx64_accelerate    conda-forge
libnghttp2                1.52.0               he2ab024_0    conda-forge
libopenblas               0.3.24          openmp_h48a4ad5_0    conda-forge
libpng                    1.6.39               ha978bb4_0    conda-forge
libprotobuf               4.23.3               h5feb325_1    conda-forge
libsodium                 1.0.18               hbcb3906_1    conda-forge
libsqlite                 3.43.0               h58db7d2_0    conda-forge
libssh2                   1.11.0               hd019ec5_0    conda-forge
libtiff                   4.5.1                hf955e92_1    conda-forge
libwebp-base              1.3.1                h0dc2134_0    conda-forge
libxcb                    1.15                 hb7f2c08_0    conda-forge
libzlib                   1.2.13               h8a1eda9_5    conda-forge
littleutils               0.2.2                      py_0    conda-forge
llvm-openmp               16.0.6               hff08bdf_0    conda-forge
logical-unification       0.4.6              pyhd8ed1ab_0    conda-forge
matplotlib                3.7.1           py311hecd8cb5_1  
matplotlib-base           3.7.1           py311h2bf763f_0    conda-forge
matplotlib-inline         0.1.6              pyhd8ed1ab_0    conda-forge
minikanren                1.0.3              pyhd8ed1ab_0    conda-forge
ml_dtypes                 0.2.0           py311hab14417_1    conda-forge
mpmath                    1.3.0              pyhd8ed1ab_0    conda-forge
multipledispatch          0.6.0                      py_0    conda-forge
munkres                   1.1.4              pyh9f0ad1d_0    conda-forge
ncurses                   6.4                  hf0c8a7f_0    conda-forge
nest-asyncio              1.5.6              pyhd8ed1ab_0    conda-forge
numpy                     1.25.2          py311hc44ba51_0    conda-forge
numpyro                   0.13.0             pyhd8ed1ab_0    conda-forge
openjpeg                  2.5.0                h13ac156_2    conda-forge
openpyxl                  3.0.10          py311h6c40b1e_0  
openssl                   3.1.4                hd75f5a5_0    conda-forge
opt_einsum                3.3.0              pyhd8ed1ab_1    conda-forge
outdated                  0.2.2              pyhd8ed1ab_0    conda-forge
packaging                 23.1               pyhd8ed1ab_0    conda-forge
pandas                    2.1.0           py311hab14417_0    conda-forge
pandas-flavor             0.6.0              pyhd8ed1ab_1    conda-forge
parso                     0.8.3              pyhd8ed1ab_0    conda-forge
patsy                     0.5.3           py311hecd8cb5_0  
pexpect                   4.8.0              pyh1a96a4e_2    conda-forge
pickleshare               0.7.5                   py_1003    conda-forge
pillow                    10.0.0          py311h7cb0e2d_0    conda-forge
pingouin                  0.5.3              pyhd8ed1ab_0    conda-forge
pip                       23.2.1             pyhd8ed1ab_0    conda-forge
platformdirs              3.10.0             pyhd8ed1ab_0    conda-forge
pooch                     1.7.0              pyha770c72_3    conda-forge
prompt-toolkit            3.0.39             pyha770c72_0    conda-forge
prompt_toolkit            3.0.39               hd8ed1ab_0    conda-forge
psutil                    5.9.5           py311h5547dcb_0    conda-forge
pthread-stubs             0.4               hc929b4f_1001    conda-forge
ptyprocess                0.7.0              pyhd3deb0d_0    conda-forge
pure_eval                 0.2.2              pyhd8ed1ab_0    conda-forge
pygments                  2.16.1             pyhd8ed1ab_0    conda-forge
pymc                      5.6.1           py311h9dd2307_0  
pyparsing                 3.1.1              pyhd8ed1ab_0    conda-forge
pysocks                   1.7.1              pyha2e5f31_6    conda-forge
pytensor                  2.12.3          py311hdb55bb0_0  
python                    3.11.5          h30d4d87_0_cpython    conda-forge
python-dateutil           2.8.2              pyhd8ed1ab_0    conda-forge
python-tzdata             2023.3             pyhd8ed1ab_0    conda-forge
python_abi                3.11                    3_cp311    conda-forge
pytz                      2023.3.post1       pyhd8ed1ab_0    conda-forge
pyzmq                     25.1.1          py311h5dacc12_0    conda-forge
re2                       2023.03.02           h096449b_0    conda-forge
readline                  8.2                  h9e318b2_1    conda-forge
requests                  2.31.0             pyhd8ed1ab_0    conda-forge
scikit-learn              1.2.2           py311hcec6c5f_1  
scipy                     1.11.3          py311h16c3c4d_1    conda-forge
seaborn                   0.12.2          py311hecd8cb5_0  
setuptools                68.1.2             pyhd8ed1ab_0    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
stack_data                0.6.2              pyhd8ed1ab_0    conda-forge
statsmodels               0.14.0          py311hb3a5e46_0  
tabulate                  0.9.0              pyhd8ed1ab_1    conda-forge
threadpoolctl             2.2.0              pyh0d69192_0  
tk                        8.6.12               h5dbffcc_0    conda-forge
toolz                     0.12.0             pyhd8ed1ab_0    conda-forge
tornado                   6.3.2           py311h6c40b1e_0  
tqdm                      4.66.1             pyhd8ed1ab_0    conda-forge
traitlets                 5.9.0              pyhd8ed1ab_0    conda-forge
typing-extensions         4.7.1                hd8ed1ab_0    conda-forge
typing_extensions         4.7.1              pyha770c72_0    conda-forge
tzdata                    2023c                h71feb2d_0    conda-forge
urllib3                   2.0.4              pyhd8ed1ab_0    conda-forge
wcwidth                   0.2.6              pyhd8ed1ab_0    conda-forge
wheel                     0.41.2             pyhd8ed1ab_0    conda-forge
xarray                    2023.8.0           pyhd8ed1ab_0    conda-forge
xarray-einstats           0.6.0              pyhd8ed1ab_0    conda-forge
xorg-libxau               1.0.11               h0dc2134_0    conda-forge
xorg-libxdmcp             1.1.3                h35c211d_0    conda-forge
xz                        5.2.6                h775f41a_0    conda-forge
zeromq                    4.3.4                he49afe7_1    conda-forge
zipp                      3.16.2             pyhd8ed1ab_0    conda-forge
zstd                      1.5.5                h829000d_0    conda-forge

but actually I also tried in a new enviroment, where I get the same error. Sampling with multiple chains throws, the fortran error, singe chain samples but incredibly slow…

Hm, that pymc version is a bit old. Can you try conda install -c conda-forge pymc=5.9.1?

thanks, this does not solve it, but installing a new environment lead to the solution. I had a warning about xcrun in the terminal, which guided me to this thread: xcrun: error: invalid active devel… | Apple Developer Forums, so I installed brew, and now everything works … I almost switched to Julia :slight_smile: (before upgrading to Sonoma, I have not updated the OS in a while, so this might have caused it ?!).

2 Likes

For youse guys running accelerate locally on Apple Silicon, what general concepts are you using to deploy to Production?

i.e. my workflow is generally develop on a MacBook and deploy to some flavor of Linux for Production. This works fine in an MKL-based world - where I use the same Makefile, Miniforge, packages etc etc on both machines. But if I move to Accelerate etc then the local dev machine will be a different architecture. My initial thought is simply to manage two different environments, but I’m open to ideas… for example potentially deploying to Amazon EC2 Mac Instances - Amazon Web Services

You could just develop and run things in a docker container using MKL, that way you can ship that directly.

Yeah - that’s what I’ve done in the end, keeps it clean. Local install for Mac-based development with a nice makefile, mamba, conga-forge, pyproject etcetc, and a slightly different monolithic devcontainer / docker based environment to give clients a development environment to be accessed via VSCode remote env, deployed onto / running from Azure Container Registry. It’s a bit of a faff, but works :slight_smile:

1 Like