Version dependant slowing down of Gaussian Mixture sampling in Ubuntu 20.04

[iavicenna]

Following the discussions here:

Why does MAP vastly outperform sample in bayesian clustering?

You can define your logp manually in a CustomDist if you want to use Censored Normals but treat the x,y pairs as non-exchangeable. However I am not sure if Censoring will make complete sense in this case, since it will be axis-aligned? What if censoring happens when x+y>b? And are you sure you are working with censoring and not truncation? Sounds interesting either way. Do you have an example of code that generates synthetic data?

I have realized that in Ubuntu when I am using pymc 5.9.1 Multivariate Gaussian is incredibly slow. I am aware that there is already an open issue in github. I have not however seen a discussion of the effect of the pymc version on this. For me the slow down only starts happening after and including 5.9. When I upgrade from 5.8.0 to 5.9.0 the amount of time taken for sampling doubles and when I pass from 5.9.0 to 5.9.1 it just becomes unreasonable, only for MV. I will dump below some info regarding this. I am on Ubuntu 20.04.

1- The code I am using is due to @jessegrabowski in the link above:

#!/usr/bin/env python3
# -*- coding: utf-8 -*-

import pymc as pm
import numpy as np
import matplotlib.pyplot as plt
import pytensor.tensor as pt
import arviz as az
from sklearn.datasets import make_blobs
from sklearn.preprocessing import StandardScaler

n_clusters = 5
data, labels = make_blobs(n_samples=1000, centers=n_clusters, random_state=10)

scaler = StandardScaler()
scaled_data = scaler.fit_transform(data)
plt.scatter(*scaled_data.T, c=labels)
coords={"cluster": np.arange(n_clusters),
        "obs_id": np.arange(data.shape[0]),
        "coord":['x', 'y']}

# When you use the ordered transform, the initial values need to be
# monotonically increasing
sorted_initvals = np.linspace(-2, 2, 5)

with pm.Model(coords=coords) as model:
    # Use alpha > 1 to prevent the model from finding sparse solutions -- we know
    # all 5 clusters should be represented in the posterior
    w = pm.Dirichlet("w", np.full(n_clusters, 10), dims=['cluster'])

    # Mean component
    x_coord = pm.Normal("x_coord", sigma=1, dims=["cluster"],
                        transform=pm.distributions.transforms.ordered,
                        initval=sorted_initvals)
    y_coord = pm.Normal('y_coord', sigma=1, dims=['cluster'])
    centroids = pm.Deterministic('centroids', pt.concatenate([x_coord[None], y_coord[None]]).T,
                                dims=['cluster', 'coord'])

    # Diagonal covariances. Could also model the full covariance matrix, but I didn't try.
    sigma = pm.HalfNormal('sigma', sigma=1, dims=['cluster', 'coord'])
    covs = [pt.diag(sigma[i]) for i in range(n_clusters)]

    # Define the mixture
    components = [pm.MvNormal.dist(mu=centroids[i], cov=covs[i]) for i in range(n_clusters)]
    y_hat = pm.Mixture("y_hat",
                       w,
                       components,
                       observed=scaled_data,
                       dims=["obs_id", 'coord'])

    idata = pm.sample(init='advi+adapt_diag')

2- In all my tests, I have installed a fresh enviroment with the following command:

conda create -n pymc_env_X_Y_Z pymc=X.Y.Z scikit-learn spyder-kernels=2.4

3- I have tested various versions starting at 5.2 to 5.9.1. In 5.9.1 but the advi initialization and sampling is unreasonably slow so I had to test sampling with just pm.sample() rather than above.

pymc 5.2

Auto-assigning NUTS sampler...
Initializing NUTS using advi+adapt_diag...
 |█████---------------| 26.98% [53965/200000 02:20<06:21 Average Loss = 1,062]9]Convergence achieved at 54000
Interrupted at 53,999 [26%]: Average Loss = 1,915.4
Multiprocess sampling (4 chains in 4 jobs)
NUTS: [w, x_coord, y_coord, sigma]
 |████████████| 100.00% [8000/8000 04:37<00:00 Sampling 4 chains, 0 divergences]Sampling 4 chains for 1_000 tune and 1_000 draw iterations (4_000 + 4_000 draws total) took 277 seconds.


pymc 5.5

Auto-assigning NUTS sampler...
Initializing NUTS using advi+adapt_diag...
 |███-----------------| 16.59% [33176/200000 01:24<07:05 Average Loss = 1,494.8]Convergence achieved at 33200
Interrupted at 33,199 [16%]: Average Loss = 2,372.7
Multiprocess sampling (4 chains in 4 jobs)
NUTS: [w, x_coord, y_coord, sigma]
 |████████████| 100.00% [8000/8000 05:40<00:00 Sampling 4 chains, 0 divergences]Sampling 4 chains for 1_000 tune and 1_000 draw iterations (4_000 + 4_000 draws total) took 340 seconds.


pymc 5.6

Auto-assigning NUTS sampler...
Initializing NUTS using advi+adapt_diag...
 |█████---------------| 26.33% [52663/200000 02:41<07:31 Average Loss = 1,062.3]Convergence achieved at 52700
Interrupted at 52,699 [26%]: Average Loss = 1,928
Multiprocess sampling (4 chains in 4 jobs)
NUTS: [w, x_coord, y_coord, sigma]
 |████████████| 100.00% [8000/8000 04:56<00:00 Sampling 4 chains, 0 divergences]Sampling 4 chains for 1_000 tune and 1_000 draw iterations (4_000 + 4_000 draws total) took 297 seconds.


version 5.7

Auto-assigning NUTS sampler...
Initializing NUTS using advi+adapt_diag...
 |█████---------------| 25.73% [51468/200000 02:39<07:39 Average Loss = 1,062.4]Convergence achieved at 51500
Interrupted at 51,499 [25%]: Average Loss = 1,947.9
Multiprocess sampling (4 chains in 4 jobs)
NUTS: [w, x_coord, y_coord, sigma]
 |████████████| 100.00% [8000/8000 05:03<00:00 Sampling 4 chains, 0 divergences]Sampling 4 chains for 1_000 tune and 1_000 draw iterations (4_000 + 4_000 draws total) took 303 seconds.


version 5.8

Auto-assigning NUTS sampler...
Initializing NUTS using advi+adapt_diag...
 |█████---------------| 25.64% [51285/200000 02:03<05:57 Average Loss = 1,062.6]Convergence achieved at 51300
Interrupted at 51,299 [25%]: Average Loss = 1,948.4
Multiprocess sampling (4 chains in 4 jobs)
NUTS: [w, x_coord, y_coord, sigma]
 |████████████| 100.00% [8000/8000 05:10<00:00 Sampling 4 chains, 0 divergences]Sampling 4 chains for 1_000 tune and 1_000 draw iterations (4_000 + 4_000 draws total) took 310 seconds.


version 5.9

Auto-assigning NUTS sampler...
Initializing NUTS using advi+adapt_diag...
 |█████---------------| 29.93% [59855/200000 02:57<06:54 Average Loss = 1,038.5]Convergence achieved at 59900
Interrupted at 59,899 [29%]: Average Loss = 1,821.7
Multiprocess sampling (4 chains in 4 jobs)
NUTS: [w, x_coord, y_coord, sigma]
 |████████████| 100.00% [8000/8000 11:06<00:00 Sampling 4 chains, 0 divergences]Sampling 4 chains for 1_000 tune and 1_000 draw iterations (4_000 + 4_000 draws total) took 666 seconds.

version 5.9.1

Auto-assigning NUTS sampler...
Initializing NUTS using advi+adapt_diag...
 |--------------------| 0.04% [80/200000 00:16<11:45:19 Average Loss = 3,777.7]

Auto-assigning NUTS sampler...
Initializing NUTS using jitter+adapt_diag...
Multiprocess sampling (4 chains in 4 jobs)
NUTS: [w, x_coord, y_coord, sigma]
 |--------------| 0.42% [34/8000 00:31<2:01:45 Sampling 4 chains, 0 divergences]

4- Here are the blas results in enviroments 5_9_1 and 5_8
5_9_1:

        Some results that you can compare against. They were 10 executions
        of gemm in float64 with matrices of shape 2000x2000 (M=N=K=2000).
        All memory layout was in C order.

        CPU tested: Xeon E5345(2.33Ghz, 8M L2 cache, 1333Mhz FSB),
                    Xeon E5430(2.66Ghz, 12M L2 cache, 1333Mhz FSB),
                    Xeon E5450(3Ghz, 12M L2 cache, 1333Mhz FSB),
                    Xeon X5560(2.8Ghz, 12M L2 cache, hyper-threads?)
                    Core 2 E8500, Core i7 930(2.8Ghz, hyper-threads enabled),
                    Core i7 950(3.07GHz, hyper-threads enabled)
                    Xeon X5550(2.67GHz, 8M l2 cache?, hyper-threads enabled)


        Libraries tested:
            * numpy with ATLAS from distribution (FC9) package (1 thread)
            * manually compiled numpy and ATLAS with 2 threads
            * goto 1.26 with 1, 2, 4 and 8 threads
            * goto2 1.13 compiled with multiple threads enabled

                          Xeon   Xeon   Xeon  Core2 i7    i7     Xeon   Xeon
        lib/nb threads    E5345  E5430  E5450 E8500 930   950    X5560  X5550

        numpy 1.3.0 blas                                                775.92s
        numpy_FC9_atlas/1 39.2s  35.0s  30.7s 29.6s 21.5s 19.60s
        goto/1            18.7s  16.1s  14.2s 13.7s 16.1s 14.67s
        numpy_MAN_atlas/2 12.0s  11.6s  10.2s  9.2s  9.0s
        goto/2             9.5s   8.1s   7.1s  7.3s  8.1s  7.4s
        goto/4             4.9s   4.4s   3.7s  -     4.1s  3.8s
        goto/8             2.7s   2.4s   2.0s  -     4.1s  3.8s
        openblas/1                                        14.04s
        openblas/2                                         7.16s
        openblas/4                                         3.71s
        openblas/8                                         3.70s
        mkl 11.0.083/1            7.97s
        mkl 10.2.2.025/1                                         13.7s
        mkl 10.2.2.025/2                                          7.6s
        mkl 10.2.2.025/4                                          4.0s
        mkl 10.2.2.025/8                                          2.0s
        goto2 1.13/1                                                     14.37s
        goto2 1.13/2                                                      7.26s
        goto2 1.13/4                                                      3.70s
        goto2 1.13/8                                                      1.94s
        goto2 1.13/16                                                     3.16s

        Test time in float32. There were 10 executions of gemm in
        float32 with matrices of shape 5000x5000 (M=N=K=5000)
        All memory layout was in C order.


        cuda version      8.0    7.5    7.0
        gpu
        M40               0.45s  0.47s
        k80               0.92s  0.96s
        K6000/NOECC       0.71s         0.69s
        P6000/NOECC       0.25s

        Titan X (Pascal)  0.28s
        GTX Titan X       0.45s  0.45s  0.47s
        GTX Titan Black   0.66s  0.64s  0.64s
        GTX 1080          0.35s
        GTX 980 Ti               0.41s
        GTX 970                  0.66s
        GTX 680                         1.57s
        GTX 750 Ti               2.01s  2.01s
        GTX 750                  2.46s  2.37s
        GTX 660                  2.32s  2.32s
        GTX 580                  2.42s
        GTX 480                  2.87s
        TX1                             7.6s (float32 storage and computation)
        GT 610                          33.5s
        
Some PyTensor flags:
    blas__ldflags= -L/home/avicenna/miniconda3/envs/pymc_env_5_9_1/lib -lcblas -lblas -lcblas -lblas -Wl,-rpath,/home/avicenna/miniconda3/envs/pymc_env_5_9_1/lib
    compiledir= /home/avicenna/.pytensor/compiledir_Linux-5.15--generic-x86_64-with-glibc2.31-x86_64-3.11.6-64
    floatX= float64
    device= cpu
Some OS information:
    sys.platform= linux
    sys.version= 3.11.6 | packaged by conda-forge | (main, Oct  3 2023, 10:40:35) [GCC 12.3.0]
    sys.prefix= /home/avicenna/miniconda3/envs/pymc_env_5_9_1
Some environment variables:
    MKL_NUM_THREADS= None
    OMP_NUM_THREADS= None
    GOTO_NUM_THREADS= None

Numpy config: (used when the PyTensor flag "blas__ldflags" is empty)
blas_info:
    libraries = ['cblas', 'blas', 'cblas', 'blas']
    library_dirs = ['/home/avicenna/miniconda3/envs/pymc_env_5_9_1/lib']
    include_dirs = ['/home/avicenna/miniconda3/envs/pymc_env_5_9_1/include']
    language = c
    define_macros = [('HAVE_CBLAS', None)]
blas_opt_info:
    define_macros = [('NO_ATLAS_INFO', 1), ('HAVE_CBLAS', None)]
    libraries = ['cblas', 'blas', 'cblas', 'blas']
    library_dirs = ['/home/avicenna/miniconda3/envs/pymc_env_5_9_1/lib']
    include_dirs = ['/home/avicenna/miniconda3/envs/pymc_env_5_9_1/include']
    language = c
lapack_info:
    libraries = ['lapack', 'blas', 'lapack', 'blas']
    library_dirs = ['/home/avicenna/miniconda3/envs/pymc_env_5_9_1/lib']
    language = f77
lapack_opt_info:
    libraries = ['lapack', 'blas', 'lapack', 'blas', 'cblas', 'blas', 'cblas', 'blas']
    library_dirs = ['/home/avicenna/miniconda3/envs/pymc_env_5_9_1/lib']
    language = c
    define_macros = [('NO_ATLAS_INFO', 1), ('HAVE_CBLAS', None)]
    include_dirs = ['/home/avicenna/miniconda3/envs/pymc_env_5_9_1/include']
Supported SIMD extensions in this NumPy install:
    baseline = SSE,SSE2,SSE3
    found = SSSE3,SSE41,POPCNT,SSE42,AVX,F16C,FMA3,AVX2,AVX512F,AVX512CD,AVX512_SKX,AVX512_CLX,AVX512_CNL,AVX512_ICL
    not found = AVX512_SPR
Numpy dot module: numpy
Numpy location: /home/avicenna/miniconda3/envs/pymc_env_5_9_1/lib/python3.11/site-packages/numpy/__init__.py
Numpy version: 1.25.2

We executed 10 calls to gemm with a and b matrices of shapes (5000, 5000) and (5000, 5000).

Total execution time: 9.80s on CPU (with direct PyTensor binding to blas).

Try to run this script a few times. Experience shows that the first time is not as fast as following calls. The difference is not big, but consistent.

5_8:

python -m pytensor.misc.check_blas

        Some results that you can compare against. They were 10 executions
        of gemm in float64 with matrices of shape 2000x2000 (M=N=K=2000).
        All memory layout was in C order.

        CPU tested: Xeon E5345(2.33Ghz, 8M L2 cache, 1333Mhz FSB),
                    Xeon E5430(2.66Ghz, 12M L2 cache, 1333Mhz FSB),
                    Xeon E5450(3Ghz, 12M L2 cache, 1333Mhz FSB),
                    Xeon X5560(2.8Ghz, 12M L2 cache, hyper-threads?)
                    Core 2 E8500, Core i7 930(2.8Ghz, hyper-threads enabled),
                    Core i7 950(3.07GHz, hyper-threads enabled)
                    Xeon X5550(2.67GHz, 8M l2 cache?, hyper-threads enabled)


        Libraries tested:
            * numpy with ATLAS from distribution (FC9) package (1 thread)
            * manually compiled numpy and ATLAS with 2 threads
            * goto 1.26 with 1, 2, 4 and 8 threads
            * goto2 1.13 compiled with multiple threads enabled

                          Xeon   Xeon   Xeon  Core2 i7    i7     Xeon   Xeon
        lib/nb threads    E5345  E5430  E5450 E8500 930   950    X5560  X5550

        numpy 1.3.0 blas                                                775.92s
        numpy_FC9_atlas/1 39.2s  35.0s  30.7s 29.6s 21.5s 19.60s
        goto/1            18.7s  16.1s  14.2s 13.7s 16.1s 14.67s
        numpy_MAN_atlas/2 12.0s  11.6s  10.2s  9.2s  9.0s
        goto/2             9.5s   8.1s   7.1s  7.3s  8.1s  7.4s
        goto/4             4.9s   4.4s   3.7s  -     4.1s  3.8s
        goto/8             2.7s   2.4s   2.0s  -     4.1s  3.8s
        openblas/1                                        14.04s
        openblas/2                                         7.16s
        openblas/4                                         3.71s
        openblas/8                                         3.70s
        mkl 11.0.083/1            7.97s
        mkl 10.2.2.025/1                                         13.7s
        mkl 10.2.2.025/2                                          7.6s
        mkl 10.2.2.025/4                                          4.0s
        mkl 10.2.2.025/8                                          2.0s
        goto2 1.13/1                                                     14.37s
        goto2 1.13/2                                                      7.26s
        goto2 1.13/4                                                      3.70s
        goto2 1.13/8                                                      1.94s
        goto2 1.13/16                                                     3.16s

        Test time in float32. There were 10 executions of gemm in
        float32 with matrices of shape 5000x5000 (M=N=K=5000)
        All memory layout was in C order.


        cuda version      8.0    7.5    7.0
        gpu
        M40               0.45s  0.47s
        k80               0.92s  0.96s
        K6000/NOECC       0.71s         0.69s
        P6000/NOECC       0.25s

        Titan X (Pascal)  0.28s
        GTX Titan X       0.45s  0.45s  0.47s
        GTX Titan Black   0.66s  0.64s  0.64s
        GTX 1080          0.35s
        GTX 980 Ti               0.41s
        GTX 970                  0.66s
        GTX 680                         1.57s
        GTX 750 Ti               2.01s  2.01s
        GTX 750                  2.46s  2.37s
        GTX 660                  2.32s  2.32s
        GTX 580                  2.42s
        GTX 480                  2.87s
        TX1                             7.6s (float32 storage and computation)
        GT 610                          33.5s
        
Some PyTensor flags:
    blas__ldflags= -L/home/avicenna/miniconda3/envs/pymc_env_5_8/lib -lcblas -lblas -lcblas -lblas
    compiledir= /home/avicenna/.pytensor/compiledir_Linux-5.15--generic-x86_64-with-glibc2.31-x86_64-3.11.6-64
    floatX= float64
    device= cpu
Some OS information:
    sys.platform= linux
    sys.version= 3.11.6 | packaged by conda-forge | (main, Oct  3 2023, 10:40:35) [GCC 12.3.0]
    sys.prefix= /home/avicenna/miniconda3/envs/pymc_env_5_8
Some environment variables:
    MKL_NUM_THREADS= None
    OMP_NUM_THREADS= None
    GOTO_NUM_THREADS= None

Numpy config: (used when the PyTensor flag "blas__ldflags" is empty)
blas_info:
    libraries = ['cblas', 'blas', 'cblas', 'blas']
    library_dirs = ['/home/avicenna/miniconda3/envs/pymc_env_5_8/lib']
    include_dirs = ['/home/avicenna/miniconda3/envs/pymc_env_5_8/include']
    language = c
    define_macros = [('HAVE_CBLAS', None)]
blas_opt_info:
    define_macros = [('NO_ATLAS_INFO', 1), ('HAVE_CBLAS', None)]
    libraries = ['cblas', 'blas', 'cblas', 'blas']
    library_dirs = ['/home/avicenna/miniconda3/envs/pymc_env_5_8/lib']
    include_dirs = ['/home/avicenna/miniconda3/envs/pymc_env_5_8/include']
    language = c
lapack_info:
    libraries = ['lapack', 'blas', 'lapack', 'blas']
    library_dirs = ['/home/avicenna/miniconda3/envs/pymc_env_5_8/lib']
    language = f77
lapack_opt_info:
    libraries = ['lapack', 'blas', 'lapack', 'blas', 'cblas', 'blas', 'cblas', 'blas']
    library_dirs = ['/home/avicenna/miniconda3/envs/pymc_env_5_8/lib']
    language = c
    define_macros = [('NO_ATLAS_INFO', 1), ('HAVE_CBLAS', None)]
    include_dirs = ['/home/avicenna/miniconda3/envs/pymc_env_5_8/include']
Supported SIMD extensions in this NumPy install:
    baseline = SSE,SSE2,SSE3
    found = SSSE3,SSE41,POPCNT,SSE42,AVX,F16C,FMA3,AVX2,AVX512F,AVX512CD,AVX512_SKX,AVX512_CLX,AVX512_CNL,AVX512_ICL
    not found = AVX512_SPR
Numpy dot module: numpy
Numpy location: /home/avicenna/miniconda3/envs/pymc_env_5_8/lib/python3.11/site-packages/numpy/__init__.py
Numpy version: 1.25.2

We executed 10 calls to gemm with a and b matrices of shapes (5000, 5000) and (5000, 5000).

Total execution time: 8.78s on CPU (with direct PyTensor binding to blas).

Try to run this script a few times. Experience shows that the first time is not as fast as following calls. The difference is not big, but consistent.

[ricardoV94]

Just to be sure, can you pass random_seed to pm.sample and confirm the time differences persist?

[iavicenna]

@ricardoV94

random_seed

I tried passing random_seed=10 in 5.9.1, 5.9 and 5.8. In version 5.9.1 time required is still unreasonable (couple hours). Comparing 5.8 to 5.9.0, time taken in 5.9.0 is still approximately double double that of 5.8.

ps: One thing I did not mention is I used pm.distributions.transforms.univariate_ordered pre 5.9 and pm.distributions.transforms.ordered post and including. That does not seem to be the source of the time difference though.

[jessegrabowski]

Does setting the environment flags mentioned in the issue do anything?

%env MKL_NUM_THREADS=1
%env OPENBLAS_NUM_THREADS=1

[iavicenna]

@jessegrabowski

%env MKL_NUM_THREADS=1
%env OPENBLAS_NUM_THREADS=1

So I have tried these variables with 5.9.1 and now it is the same as 5.9.0 which is roughly 2-3x the amount of time it takes in 5.8 (without setting the enviroment variables in 5.8 though).

[ricardoV94]

Is it installing the same version of numpy in the older versions of PyMC?

[iavicenna]

Seems so, in both cases it is 1.25.2. libblas versions also look the same. Here are the list of packages in 5.9.1 and 5.8 in case it helps:

5.9.1:

# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       2_gnu    conda-forge
arviz                     0.16.1             pyhd8ed1ab_1    conda-forge
asttokens                 2.4.1              pyhd8ed1ab_0    conda-forge
atk-1.0                   2.38.0               hd4edc92_1    conda-forge
backcall                  0.2.0              pyh9f0ad1d_0    conda-forge
backports                 1.0                pyhd8ed1ab_3    conda-forge
backports.functools_lru_cache 1.6.5              pyhd8ed1ab_0    conda-forge
brotli                    1.1.0                hd590300_1    conda-forge
brotli-bin                1.1.0                hd590300_1    conda-forge
bzip2                     1.0.8                h7f98852_4    conda-forge
c-ares                    1.20.1               hd590300_1    conda-forge
ca-certificates           2023.7.22            hbcca054_0    conda-forge
cached-property           1.5.2                hd8ed1ab_1    conda-forge
cached_property           1.5.2              pyha770c72_1    conda-forge
cachetools                5.3.2              pyhd8ed1ab_0    conda-forge
cairo                     1.18.0               h3faef2a_0    conda-forge
certifi                   2023.7.22          pyhd8ed1ab_0    conda-forge
cloudpickle               3.0.0              pyhd8ed1ab_0    conda-forge
comm                      0.1.4              pyhd8ed1ab_0    conda-forge
cons                      0.4.6              pyhd8ed1ab_0    conda-forge
contourpy                 1.1.1           py311h9547e67_1    conda-forge
cycler                    0.12.1             pyhd8ed1ab_0    conda-forge
debugpy                   1.8.0           py311hb755f60_1    conda-forge
decorator                 5.1.1              pyhd8ed1ab_0    conda-forge
etuples                   0.3.9              pyhd8ed1ab_0    conda-forge
exceptiongroup            1.1.3              pyhd8ed1ab_0    conda-forge
executing                 2.0.1              pyhd8ed1ab_0    conda-forge
expat                     2.5.0                hcb278e6_1    conda-forge
fastprogress              1.0.3              pyhd8ed1ab_0    conda-forge
filelock                  3.13.1             pyhd8ed1ab_0    conda-forge
font-ttf-dejavu-sans-mono 2.37                 hab24e00_0    conda-forge
font-ttf-inconsolata      3.000                h77eed37_0    conda-forge
font-ttf-source-code-pro  2.038                h77eed37_0    conda-forge
font-ttf-ubuntu           0.83                 hab24e00_0    conda-forge
fontconfig                2.14.2               h14ed4e7_0    conda-forge
fonts-conda-ecosystem     1                             0    conda-forge
fonts-conda-forge         1                             0    conda-forge
fonttools                 4.43.1          py311h459d7ec_0    conda-forge
freetype                  2.12.1               h267a509_2    conda-forge
fribidi                   1.0.10               h36c2ea0_0    conda-forge
gdk-pixbuf                2.42.10              h829c605_4    conda-forge
gettext                   0.21.1               h27087fc_0    conda-forge
giflib                    5.2.1                h0b41bf4_3    conda-forge
graphite2                 1.3.13            h58526e2_1001    conda-forge
graphviz                  8.1.0                h28d9a01_0    conda-forge
gtk2                      2.24.33              h90689f9_2    conda-forge
gts                       0.7.6                h977cf35_4    conda-forge
h5netcdf                  1.2.0              pyhd8ed1ab_0    conda-forge
h5py                      3.10.0          nompi_py311h3839ddf_100    conda-forge
harfbuzz                  8.2.1                h3d44ed6_0    conda-forge
hdf5                      1.14.2          nompi_h4f84152_100    conda-forge
icu                       73.2                 h59595ed_0    conda-forge
importlib-metadata        6.8.0              pyha770c72_0    conda-forge
importlib_metadata        6.8.0                hd8ed1ab_0    conda-forge
ipykernel                 6.26.0             pyhf8b6a83_0    conda-forge
ipython                   8.16.1             pyh0d859eb_0    conda-forge
jedi                      0.19.1             pyhd8ed1ab_0    conda-forge
joblib                    1.3.2              pyhd8ed1ab_0    conda-forge
jupyter_client            8.5.0              pyhd8ed1ab_0    conda-forge
jupyter_core              5.5.0           py311h38be061_0    conda-forge
keyutils                  1.6.1                h166bdaf_0    conda-forge
kiwisolver                1.4.5           py311h9547e67_1    conda-forge
krb5                      1.21.2               h659d440_0    conda-forge
lcms2                     2.15                 hb7c19ff_3    conda-forge
ld_impl_linux-64          2.40                 h41732ed_0    conda-forge
lerc                      4.0.0                h27087fc_0    conda-forge
libaec                    1.1.2                h59595ed_1    conda-forge
libblas                   3.9.0           19_linux64_openblas    conda-forge
libbrotlicommon           1.1.0                hd590300_1    conda-forge
libbrotlidec              1.1.0                hd590300_1    conda-forge
libbrotlienc              1.1.0                hd590300_1    conda-forge
libcblas                  3.9.0           19_linux64_openblas    conda-forge
libcurl                   8.4.0                hca28451_0    conda-forge
libdeflate                1.19                 hd590300_0    conda-forge
libedit                   3.1.20191231         he28a2e2_2    conda-forge
libev                     4.33                 h516909a_1    conda-forge
libexpat                  2.5.0                hcb278e6_1    conda-forge
libffi                    3.4.2                h7f98852_5    conda-forge
libgcc-ng                 13.2.0               h807b86a_2    conda-forge
libgd                     2.3.3                h119a65a_9    conda-forge
libgfortran-ng            13.2.0               h69a702a_2    conda-forge
libgfortran5              13.2.0               ha4646dd_2    conda-forge
libglib                   2.78.0               hebfc3b9_0    conda-forge
libgomp                   13.2.0               h807b86a_2    conda-forge
libiconv                  1.17                 h166bdaf_0    conda-forge
libjpeg-turbo             3.0.0                hd590300_1    conda-forge
liblapack                 3.9.0           19_linux64_openblas    conda-forge
libnghttp2                1.55.1               h47da74e_0    conda-forge
libnsl                    2.0.1                hd590300_0    conda-forge
libopenblas               0.3.24          pthreads_h413a1c8_0    conda-forge
libpng                    1.6.39               h753d276_0    conda-forge
librsvg                   2.56.3               h98fae49_0    conda-forge
libsodium                 1.0.18               h36c2ea0_1    conda-forge
libsqlite                 3.43.2               h2797004_0    conda-forge
libssh2                   1.11.0               h0841786_0    conda-forge
libstdcxx-ng              13.2.0               h7e041cc_2    conda-forge
libtiff                   4.6.0                ha9c0a0a_2    conda-forge
libtool                   2.4.7                h27087fc_0    conda-forge
libuuid                   2.38.1               h0b41bf4_0    conda-forge
libwebp                   1.3.2                h658648e_1    conda-forge
libwebp-base              1.3.2                hd590300_0    conda-forge
libxcb                    1.15                 h0b41bf4_0    conda-forge
libxml2                   2.11.5               h232c23b_1    conda-forge
libzlib                   1.2.13               hd590300_5    conda-forge
logical-unification       0.4.6              pyhd8ed1ab_0    conda-forge
matplotlib-base           3.8.0           py311h54ef318_2    conda-forge
matplotlib-inline         0.1.6              pyhd8ed1ab_0    conda-forge
minikanren                1.0.3              pyhd8ed1ab_0    conda-forge
multipledispatch          0.6.0                      py_0    conda-forge
munkres                   1.1.4              pyh9f0ad1d_0    conda-forge
ncurses                   6.4                  h59595ed_2    conda-forge
nest-asyncio              1.5.8              pyhd8ed1ab_0    conda-forge
numpy                     1.25.2          py311h64a7726_0    conda-forge
openjpeg                  2.5.0                h488ebb8_3    conda-forge
openssl                   3.1.4                hd590300_0    conda-forge
packaging                 23.2               pyhd8ed1ab_0    conda-forge
pandas                    2.1.2           py311h320fe9a_0    conda-forge
pango                     1.50.14              ha41ecd1_2    conda-forge
parso                     0.8.3              pyhd8ed1ab_0    conda-forge
pcre2                     10.40                hc3806b6_0    conda-forge
pexpect                   4.8.0              pyh1a96a4e_2    conda-forge
pickleshare               0.7.5                   py_1003    conda-forge
pillow                    10.1.0          py311ha6c5da5_0    conda-forge
pip                       23.3.1             pyhd8ed1ab_0    conda-forge
pixman                    0.42.2               h59595ed_0    conda-forge
platformdirs              3.11.0             pyhd8ed1ab_0    conda-forge
prompt-toolkit            3.0.39             pyha770c72_0    conda-forge
prompt_toolkit            3.0.39               hd8ed1ab_0    conda-forge
psutil                    5.9.5           py311h459d7ec_1    conda-forge
pthread-stubs             0.4               h36c2ea0_1001    conda-forge
ptyprocess                0.7.0              pyhd3deb0d_0    conda-forge
pure_eval                 0.2.2              pyhd8ed1ab_0    conda-forge
pygments                  2.16.1             pyhd8ed1ab_0    conda-forge
pymc                      5.9.1                hd8ed1ab_0    conda-forge
pymc-base                 5.9.1              pyhd8ed1ab_0    conda-forge
pyparsing                 3.1.1              pyhd8ed1ab_0    conda-forge
pytensor                  2.13.1          py311ha02d727_0  
pytensor-base             2.17.3          py311h320fe9a_0    conda-forge
python                    3.11.6          hab00c5b_0_cpython    conda-forge
python-dateutil           2.8.2              pyhd8ed1ab_0    conda-forge
python-graphviz           0.20.1             pyh22cad53_0    conda-forge
python-tzdata             2023.3             pyhd8ed1ab_0    conda-forge
python_abi                3.11                    4_cp311    conda-forge
pytz                      2023.3.post1       pyhd8ed1ab_0    conda-forge
pyzmq                     25.1.1          py311h34ded2d_2    conda-forge
readline                  8.2                  h8228510_1    conda-forge
scikit-learn              1.3.2           py311hc009520_1    conda-forge
scipy                     1.11.3          py311h64a7726_1    conda-forge
setuptools                68.2.2             pyhd8ed1ab_0    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
spyder-kernels            2.4.4           unix_pyh707e725_0    conda-forge
stack_data                0.6.2              pyhd8ed1ab_0    conda-forge
threadpoolctl             3.2.0              pyha21a80b_0    conda-forge
tk                        8.6.13               h2797004_0    conda-forge
toolz                     0.12.0             pyhd8ed1ab_0    conda-forge
tornado                   6.3.3           py311h459d7ec_1    conda-forge
traitlets                 5.13.0             pyhd8ed1ab_0    conda-forge
typing-extensions         4.8.0                hd8ed1ab_0    conda-forge
typing_extensions         4.8.0              pyha770c72_0    conda-forge
tzdata                    2023c                h71feb2d_0    conda-forge
wcwidth                   0.2.9              pyhd8ed1ab_0    conda-forge
wheel                     0.41.3             pyhd8ed1ab_0    conda-forge
wurlitzer                 3.0.3              pyhd8ed1ab_0    conda-forge
xarray                    2023.10.1          pyhd8ed1ab_0    conda-forge
xarray-einstats           0.6.0              pyhd8ed1ab_0    conda-forge
xorg-kbproto              1.0.7             h7f98852_1002    conda-forge
xorg-libice               1.1.1                hd590300_0    conda-forge
xorg-libsm                1.2.4                h7391055_0    conda-forge
xorg-libx11               1.8.7                h8ee46fc_0    conda-forge
xorg-libxau               1.0.11               hd590300_0    conda-forge
xorg-libxdmcp             1.1.3                h7f98852_0    conda-forge
xorg-libxext              1.3.4                h0b41bf4_2    conda-forge
xorg-libxrender           0.9.11               hd590300_0    conda-forge
xorg-renderproto          0.11.1            h7f98852_1002    conda-forge
xorg-xextproto            7.3.0             h0b41bf4_1003    conda-forge
xorg-xproto               7.0.31            h7f98852_1007    conda-forge
xz                        5.2.6                h166bdaf_0    conda-forge
zeromq                    4.3.5                h59595ed_0    conda-forge
zipp                      3.17.0             pyhd8ed1ab_0    conda-forge
zlib                      1.2.13               hd590300_5    conda-forge
zstd                      1.5.5                hfc55251_0    conda-forge

5.8

# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                  2_kmp_llvm    conda-forge
arviz                     0.16.1             pyhd8ed1ab_1    conda-forge
asttokens                 2.4.1              pyhd8ed1ab_0    conda-forge
backcall                  0.2.0              pyh9f0ad1d_0    conda-forge
backports                 1.0                pyhd8ed1ab_3    conda-forge
backports.functools_lru_cache 1.6.5              pyhd8ed1ab_0    conda-forge
binutils_impl_linux-64    2.40                 hf600244_0    conda-forge
binutils_linux-64         2.40                 hbdbef99_2    conda-forge
blas                      2.119                  openblas    conda-forge
blas-devel                3.9.0           19_linux64_openblas    conda-forge
brotli                    1.1.0                hd590300_1    conda-forge
brotli-bin                1.1.0                hd590300_1    conda-forge
bzip2                     1.0.8                h7f98852_4    conda-forge
c-ares                    1.20.1               hd590300_1    conda-forge
ca-certificates           2023.7.22            hbcca054_0    conda-forge
cached-property           1.5.2                hd8ed1ab_1    conda-forge
cached_property           1.5.2              pyha770c72_1    conda-forge
cachetools                5.3.2              pyhd8ed1ab_0    conda-forge
certifi                   2023.7.22          pyhd8ed1ab_0    conda-forge
cloudpickle               3.0.0              pyhd8ed1ab_0    conda-forge
comm                      0.1.4              pyhd8ed1ab_0    conda-forge
cons                      0.4.6              pyhd8ed1ab_0    conda-forge
contourpy                 1.1.1           py311h9547e67_1    conda-forge
cycler                    0.12.1             pyhd8ed1ab_0    conda-forge
debugpy                   1.8.0           py311hb755f60_1    conda-forge
decorator                 5.1.1              pyhd8ed1ab_0    conda-forge
etuples                   0.3.9              pyhd8ed1ab_0    conda-forge
exceptiongroup            1.1.3              pyhd8ed1ab_0    conda-forge
executing                 2.0.1              pyhd8ed1ab_0    conda-forge
fastprogress              1.0.3              pyhd8ed1ab_0    conda-forge
filelock                  3.13.1             pyhd8ed1ab_0    conda-forge
fonttools                 4.43.1          py311h459d7ec_0    conda-forge
freetype                  2.12.1               h267a509_2    conda-forge
gcc                       12.3.0               h8d2909c_2    conda-forge
gcc_impl_linux-64         12.3.0               he2b93b0_2    conda-forge
gcc_linux-64              12.3.0               h76fc315_2    conda-forge
gxx                       12.3.0               h8d2909c_2    conda-forge
gxx_impl_linux-64         12.3.0               he2b93b0_2    conda-forge
gxx_linux-64              12.3.0               h8a814eb_2    conda-forge
h5netcdf                  1.2.0              pyhd8ed1ab_0    conda-forge
h5py                      3.10.0          nompi_py311h3839ddf_100    conda-forge
hdf5                      1.14.2          nompi_h4f84152_100    conda-forge
icu                       73.2                 h59595ed_0    conda-forge
importlib-metadata        6.8.0              pyha770c72_0    conda-forge
importlib_metadata        6.8.0                hd8ed1ab_0    conda-forge
ipykernel                 6.26.0             pyhf8b6a83_0    conda-forge
ipython                   8.16.1             pyh0d859eb_0    conda-forge
jedi                      0.19.1             pyhd8ed1ab_0    conda-forge
joblib                    1.3.2              pyhd8ed1ab_0    conda-forge
jupyter_client            8.5.0              pyhd8ed1ab_0    conda-forge
jupyter_core              5.5.0           py311h38be061_0    conda-forge
kernel-headers_linux-64   2.6.32              he073ed8_16    conda-forge
keyutils                  1.6.1                h166bdaf_0    conda-forge
kiwisolver                1.4.5           py311h9547e67_1    conda-forge
krb5                      1.21.2               h659d440_0    conda-forge
lcms2                     2.15                 hb7c19ff_3    conda-forge
ld_impl_linux-64          2.40                 h41732ed_0    conda-forge
lerc                      4.0.0                h27087fc_0    conda-forge
libaec                    1.1.2                h59595ed_1    conda-forge
libblas                   3.9.0           19_linux64_openblas    conda-forge
libbrotlicommon           1.1.0                hd590300_1    conda-forge
libbrotlidec              1.1.0                hd590300_1    conda-forge
libbrotlienc              1.1.0                hd590300_1    conda-forge
libcblas                  3.9.0           19_linux64_openblas    conda-forge
libcurl                   8.4.0                hca28451_0    conda-forge
libdeflate                1.19                 hd590300_0    conda-forge
libedit                   3.1.20191231         he28a2e2_2    conda-forge
libev                     4.33                 h516909a_1    conda-forge
libexpat                  2.5.0                hcb278e6_1    conda-forge
libffi                    3.4.2                h7f98852_5    conda-forge
libgcc-devel_linux-64     12.3.0               h8bca6fd_2    conda-forge
libgcc-ng                 13.2.0               h807b86a_2    conda-forge
libgfortran-ng            13.2.0               h69a702a_2    conda-forge
libgfortran5              13.2.0               ha4646dd_2    conda-forge
libgomp                   13.2.0               h807b86a_2    conda-forge
libhwloc                  2.9.3           default_h554bfaf_1009    conda-forge
libiconv                  1.17                 h166bdaf_0    conda-forge
libjpeg-turbo             3.0.0                hd590300_1    conda-forge
liblapack                 3.9.0           19_linux64_openblas    conda-forge
liblapacke                3.9.0           19_linux64_openblas    conda-forge
libnghttp2                1.55.1               h47da74e_0    conda-forge
libnsl                    2.0.1                hd590300_0    conda-forge
libopenblas               0.3.24          pthreads_h413a1c8_0    conda-forge
libpng                    1.6.39               h753d276_0    conda-forge
libsanitizer              12.3.0               h0f45ef3_2    conda-forge
libsodium                 1.0.18               h36c2ea0_1    conda-forge
libsqlite                 3.43.2               h2797004_0    conda-forge
libssh2                   1.11.0               h0841786_0    conda-forge
libstdcxx-devel_linux-64  12.3.0               h8bca6fd_2    conda-forge
libstdcxx-ng              13.2.0               h7e041cc_2    conda-forge
libtiff                   4.6.0                ha9c0a0a_2    conda-forge
libuuid                   2.38.1               h0b41bf4_0    conda-forge
libwebp-base              1.3.2                hd590300_0    conda-forge
libxcb                    1.15                 h0b41bf4_0    conda-forge
libxml2                   2.11.5               h232c23b_1    conda-forge
libzlib                   1.2.13               hd590300_5    conda-forge
llvm-openmp               17.0.3               h4dfa4b3_0    conda-forge
logical-unification       0.4.6              pyhd8ed1ab_0    conda-forge
matplotlib-base           3.8.0           py311h54ef318_2    conda-forge
matplotlib-inline         0.1.6              pyhd8ed1ab_0    conda-forge
minikanren                1.0.3              pyhd8ed1ab_0    conda-forge
mkl                       2022.2.1         h84fe81f_16997    conda-forge
mkl-service               2.4.0           py311hb711fc7_0    conda-forge
multipledispatch          0.6.0                      py_0    conda-forge
munkres                   1.1.4              pyh9f0ad1d_0    conda-forge
ncurses                   6.4                  h59595ed_2    conda-forge
nest-asyncio              1.5.8              pyhd8ed1ab_0    conda-forge
numpy                     1.25.2          py311h64a7726_0    conda-forge
openblas                  0.3.24          pthreads_h7a3da1a_0    conda-forge
openjpeg                  2.5.0                h488ebb8_3    conda-forge
openssl                   3.1.4                hd590300_0    conda-forge
packaging                 23.2               pyhd8ed1ab_0    conda-forge
pandas                    2.1.2           py311h320fe9a_0    conda-forge
parso                     0.8.3              pyhd8ed1ab_0    conda-forge
pexpect                   4.8.0              pyh1a96a4e_2    conda-forge
pickleshare               0.7.5                   py_1003    conda-forge
pillow                    10.1.0          py311ha6c5da5_0    conda-forge
pip                       23.3.1             pyhd8ed1ab_0    conda-forge
platformdirs              3.11.0             pyhd8ed1ab_0    conda-forge
prompt-toolkit            3.0.39             pyha770c72_0    conda-forge
prompt_toolkit            3.0.39               hd8ed1ab_0    conda-forge
psutil                    5.9.5           py311h459d7ec_1    conda-forge
pthread-stubs             0.4               h36c2ea0_1001    conda-forge
ptyprocess                0.7.0              pyhd3deb0d_0    conda-forge
pure_eval                 0.2.2              pyhd8ed1ab_0    conda-forge
pygments                  2.16.1             pyhd8ed1ab_0    conda-forge
pymc                      5.8.0                hd8ed1ab_0    conda-forge
pymc-base                 5.8.0              pyhd8ed1ab_0    conda-forge
pyparsing                 3.1.1              pyhd8ed1ab_0    conda-forge
pytensor                  2.15.0          py311hb755f60_0    conda-forge
pytensor-base             2.15.0          py311h320fe9a_0    conda-forge
python                    3.11.6          hab00c5b_0_cpython    conda-forge
python-dateutil           2.8.2              pyhd8ed1ab_0    conda-forge
python-tzdata             2023.3             pyhd8ed1ab_0    conda-forge
python_abi                3.11                    4_cp311    conda-forge
pytz                      2023.3.post1       pyhd8ed1ab_0    conda-forge
pyzmq                     25.1.1          py311h34ded2d_2    conda-forge
readline                  8.2                  h8228510_1    conda-forge
scikit-learn              1.3.2           py311hc009520_1    conda-forge
scipy                     1.11.3          py311h64a7726_1    conda-forge
setuptools                68.2.2             pyhd8ed1ab_0    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
spyder-kernels            2.4.4           unix_pyh707e725_0    conda-forge
stack_data                0.6.2              pyhd8ed1ab_0    conda-forge
sysroot_linux-64          2.12                he073ed8_16    conda-forge
tbb                       2021.10.0            h00ab1b0_2    conda-forge
threadpoolctl             3.2.0              pyha21a80b_0    conda-forge
tk                        8.6.13               h2797004_0    conda-forge
toolz                     0.12.0             pyhd8ed1ab_0    conda-forge
tornado                   6.3.3           py311h459d7ec_1    conda-forge
traitlets                 5.13.0             pyhd8ed1ab_0    conda-forge
typing-extensions         4.8.0                hd8ed1ab_0    conda-forge
typing_extensions         4.8.0              pyha770c72_0    conda-forge
tzdata                    2023c                h71feb2d_0    conda-forge
wcwidth                   0.2.9              pyhd8ed1ab_0    conda-forge
wheel                     0.41.3             pyhd8ed1ab_0    conda-forge
wurlitzer                 3.0.3              pyhd8ed1ab_0    conda-forge
xarray                    2023.10.1          pyhd8ed1ab_0    conda-forge
xarray-einstats           0.6.0              pyhd8ed1ab_0    conda-forge
xorg-libxau               1.0.11               hd590300_0    conda-forge
xorg-libxdmcp             1.1.3                h7f98852_0    conda-forge
xz                        5.2.6                h166bdaf_0    conda-forge
zeromq                    4.3.5                h59595ed_0    conda-forge
zipp                      3.17.0             pyhd8ed1ab_0    conda-forge
zstd                      1.5.5                hfc55251_0    conda-forge

[iavicenna]

Here is what conda compare returns (running one is 5.8):

_openmp_mutex found but mismatch. Specification pkg: _openmp_mutex==4.5=2_gnu, Running pkg: _openmp_mutex==4.5=2_kmp_llvm
atk-1.0 not found
cairo not found
expat not found
font-ttf-dejavu-sans-mono not found
font-ttf-inconsolata not found
font-ttf-source-code-pro not found
font-ttf-ubuntu not found
fontconfig not found
fonts-conda-ecosystem not found
fonts-conda-forge not found
fribidi not found
gdk-pixbuf not found
gettext not found
giflib not found
graphite2 not found
graphviz not found
gtk2 not found
gts not found
harfbuzz not found
libgd not found
libglib not found
librsvg not found
libtool not found
libwebp not found
pango not found
pcre2 not found
pixman not found
pymc found but mismatch. Specification pkg: pymc==5.9.1=hd8ed1ab_0, Running pkg: pymc==5.8.0=hd8ed1ab_0
pymc-base found but mismatch. Specification pkg: pymc-base==5.9.1=pyhd8ed1ab_0, Running pkg: pymc-base==5.8.0=pyhd8ed1ab_0
pytensor found but mismatch. Specification pkg: pytensor==2.13.1=py311ha02d727_0, Running pkg: pytensor==2.15.0=py311hb755f60_0
pytensor-base found but mismatch. Specification pkg: pytensor-base==2.17.3=py311h320fe9a_0, Running pkg: pytensor-base==2.15.0=py311h320fe9a_0
python-graphviz not found
xorg-kbproto not found
xorg-libice not found
xorg-libsm not found
xorg-libx11 not found
xorg-libxext not found
xorg-libxrender not found
xorg-renderproto not found
xorg-xextproto not found
xorg-xproto not found
zlib not found

Possible related differences are pytensor and openmp_mutex. Maybe likely the latter given that issue seems to be threading related? In light of this, I also compare 5.9 to 5.9.1 (because eventhough 5.9 slows down, 5.9.1 just completely breaks):

_openmp_mutex found but mismatch. Specification pkg: _openmp_mutex==4.5=2_gnu, Running pkg: _openmp_mutex==4.5=2_kmp_llvm
pymc found but mismatch. Specification pkg: pymc==5.9.1=hd8ed1ab_0, Running pkg: pymc==5.9.0=hd8ed1ab_0
pymc-base found but mismatch. Specification pkg: pymc-base==5.9.1=pyhd8ed1ab_0, Running pkg: pymc-base==5.9.0=pyhd8ed1ab_0
pytensor found but mismatch. Specification pkg: pytensor==2.13.1=py311ha02d727_0, Running pkg: pytensor==2.17.3=py311hb755f60_0

and when I compare 5.8 to 5.9 I get (not showing the fonts and such)

pymc found but mismatch. Specification pkg: pymc==5.9.0=hd8ed1ab_0, Running pkg: pymc==5.8.0=hd8ed1ab_0
pymc-base found but mismatch. Specification pkg: pymc-base==5.9.0=pyhd8ed1ab_0, Running pkg: pymc-base==5.8.0=pyhd8ed1ab_0
pytensor found but mismatch. Specification pkg: pytensor==2.17.3=py311hb755f60_0, Running pkg: pytensor==2.15.0=py311hb755f60_0
pytensor-base found but mismatch. Specification pkg: pytensor-base==2.17.3=py311h320fe9a_0, Running pkg: pytensor-base==2.15.0=py311h320fe9a_0

So simplest explanation is the doubling of time from 5.8 → 5.9 might be pytensor_ base related and
breaking from 5.9 → 5.9.1 might be _open_mutex related?

[jessegrabowski]

I don’t see mkl and mkl-service in 5.9.1. Those are relevant for high-performance linear algebra (assuming you’re on an intel processor).

Can you try running conda install "libblas=*=*mkl" in your 5.9.1 env? This helped another user here,

[iavicenna]

Good spot (I am on intel), though after installing this, with or without advi sampling is still extremely slow (to a point it seems to get stuck). I did the installation as suggested.

Here is what is changed:

## Package Plan ##

  environment location: /home/avicenna/miniconda3/envs/pymc_env_5_9_1

  added / updated specs:
    - libblas[build=*mkl]


The following NEW packages will be INSTALLED:

  intel-openmp       pkgs/main/linux-64::intel-openmp-2021.4.0-h06a4308_3561 
  mkl                pkgs/main/linux-64::mkl-2021.4.0-h06a4308_640 

The following packages will be DOWNGRADED:

  libblas                         3.9.0-19_linux64_openblas --> 3.9.0-12_linux64_mkl 
  libcblas                        3.9.0-19_linux64_openblas --> 3.9.0-12_linux64_mkl 
  liblapack                       3.9.0-19_linux64_openblas --> 3.9.0-12_linux64_mkl 

and the current env

# packages in environment at /home/avicenna/miniconda3/envs/pymc_env_5_9_1:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       2_gnu    conda-forge
arviz                     0.16.1             pyhd8ed1ab_1    conda-forge
asttokens                 2.4.1              pyhd8ed1ab_0    conda-forge
atk-1.0                   2.38.0               hd4edc92_1    conda-forge
backcall                  0.2.0              pyh9f0ad1d_0    conda-forge
backports                 1.0                pyhd8ed1ab_3    conda-forge
backports.functools_lru_cache 1.6.5              pyhd8ed1ab_0    conda-forge
brotli                    1.1.0                hd590300_1    conda-forge
brotli-bin                1.1.0                hd590300_1    conda-forge
bzip2                     1.0.8                h7f98852_4    conda-forge
c-ares                    1.20.1               hd590300_1    conda-forge
ca-certificates           2023.7.22            hbcca054_0    conda-forge
cached-property           1.5.2                hd8ed1ab_1    conda-forge
cached_property           1.5.2              pyha770c72_1    conda-forge
cachetools                5.3.2              pyhd8ed1ab_0    conda-forge
cairo                     1.18.0               h3faef2a_0    conda-forge
certifi                   2023.7.22          pyhd8ed1ab_0    conda-forge
cloudpickle               3.0.0              pyhd8ed1ab_0    conda-forge
comm                      0.1.4              pyhd8ed1ab_0    conda-forge
cons                      0.4.6              pyhd8ed1ab_0    conda-forge
contourpy                 1.1.1           py311h9547e67_1    conda-forge
cycler                    0.12.1             pyhd8ed1ab_0    conda-forge
debugpy                   1.8.0           py311hb755f60_1    conda-forge
decorator                 5.1.1              pyhd8ed1ab_0    conda-forge
etuples                   0.3.9              pyhd8ed1ab_0    conda-forge
exceptiongroup            1.1.3              pyhd8ed1ab_0    conda-forge
executing                 2.0.1              pyhd8ed1ab_0    conda-forge
expat                     2.5.0                hcb278e6_1    conda-forge
fastprogress              1.0.3              pyhd8ed1ab_0    conda-forge
filelock                  3.13.1             pyhd8ed1ab_0    conda-forge
font-ttf-dejavu-sans-mono 2.37                 hab24e00_0    conda-forge
font-ttf-inconsolata      3.000                h77eed37_0    conda-forge
font-ttf-source-code-pro  2.038                h77eed37_0    conda-forge
font-ttf-ubuntu           0.83                 hab24e00_0    conda-forge
fontconfig                2.14.2               h14ed4e7_0    conda-forge
fonts-conda-ecosystem     1                             0    conda-forge
fonts-conda-forge         1                             0    conda-forge
fonttools                 4.43.1          py311h459d7ec_0    conda-forge
freetype                  2.12.1               h267a509_2    conda-forge
fribidi                   1.0.10               h36c2ea0_0    conda-forge
gdk-pixbuf                2.42.10              h829c605_4    conda-forge
gettext                   0.21.1               h27087fc_0    conda-forge
giflib                    5.2.1                h0b41bf4_3    conda-forge
graphite2                 1.3.13            h58526e2_1001    conda-forge
graphviz                  8.1.0                h28d9a01_0    conda-forge
gtk2                      2.24.33              h90689f9_2    conda-forge
gts                       0.7.6                h977cf35_4    conda-forge
h5netcdf                  1.2.0              pyhd8ed1ab_0    conda-forge
h5py                      3.10.0          nompi_py311h3839ddf_100    conda-forge
harfbuzz                  8.2.1                h3d44ed6_0    conda-forge
hdf5                      1.14.2          nompi_h4f84152_100    conda-forge
icu                       73.2                 h59595ed_0    conda-forge
importlib-metadata        6.8.0              pyha770c72_0    conda-forge
importlib_metadata        6.8.0                hd8ed1ab_0    conda-forge
intel-openmp              2021.4.0          h06a4308_3561  
ipykernel                 6.26.0             pyhf8b6a83_0    conda-forge
ipython                   8.16.1             pyh0d859eb_0    conda-forge
jedi                      0.19.1             pyhd8ed1ab_0    conda-forge
joblib                    1.3.2              pyhd8ed1ab_0    conda-forge
jupyter_client            8.5.0              pyhd8ed1ab_0    conda-forge
jupyter_core              5.5.0           py311h38be061_0    conda-forge
keyutils                  1.6.1                h166bdaf_0    conda-forge
kiwisolver                1.4.5           py311h9547e67_1    conda-forge
krb5                      1.21.2               h659d440_0    conda-forge
lcms2                     2.15                 hb7c19ff_3    conda-forge
ld_impl_linux-64          2.40                 h41732ed_0    conda-forge
lerc                      4.0.0                h27087fc_0    conda-forge
libaec                    1.1.2                h59595ed_1    conda-forge
libblas                   3.9.0            12_linux64_mkl    conda-forge
libbrotlicommon           1.1.0                hd590300_1    conda-forge
libbrotlidec              1.1.0                hd590300_1    conda-forge
libbrotlienc              1.1.0                hd590300_1    conda-forge
libcblas                  3.9.0            12_linux64_mkl    conda-forge
libcurl                   8.4.0                hca28451_0    conda-forge
libdeflate                1.19                 hd590300_0    conda-forge
libedit                   3.1.20191231         he28a2e2_2    conda-forge
libev                     4.33                 h516909a_1    conda-forge
libexpat                  2.5.0                hcb278e6_1    conda-forge
libffi                    3.4.2                h7f98852_5    conda-forge
libgcc-ng                 13.2.0               h807b86a_2    conda-forge
libgd                     2.3.3                h119a65a_9    conda-forge
libgfortran-ng            13.2.0               h69a702a_2    conda-forge
libgfortran5              13.2.0               ha4646dd_2    conda-forge
libglib                   2.78.0               hebfc3b9_0    conda-forge
libgomp                   13.2.0               h807b86a_2    conda-forge
libiconv                  1.17                 h166bdaf_0    conda-forge
libjpeg-turbo             3.0.0                hd590300_1    conda-forge
liblapack                 3.9.0            12_linux64_mkl    conda-forge
libnghttp2                1.55.1               h47da74e_0    conda-forge
libnsl                    2.0.1                hd590300_0    conda-forge
libopenblas               0.3.24          pthreads_h413a1c8_0    conda-forge
libpng                    1.6.39               h753d276_0    conda-forge
librsvg                   2.56.3               h98fae49_0    conda-forge
libsodium                 1.0.18               h36c2ea0_1    conda-forge
libsqlite                 3.43.2               h2797004_0    conda-forge
libssh2                   1.11.0               h0841786_0    conda-forge
libstdcxx-ng              13.2.0               h7e041cc_2    conda-forge
libtiff                   4.6.0                ha9c0a0a_2    conda-forge
libtool                   2.4.7                h27087fc_0    conda-forge
libuuid                   2.38.1               h0b41bf4_0    conda-forge
libwebp                   1.3.2                h658648e_1    conda-forge
libwebp-base              1.3.2                hd590300_0    conda-forge
libxcb                    1.15                 h0b41bf4_0    conda-forge
libxml2                   2.11.5               h232c23b_1    conda-forge
libzlib                   1.2.13               hd590300_5    conda-forge
logical-unification       0.4.6              pyhd8ed1ab_0    conda-forge
matplotlib-base           3.8.0           py311h54ef318_2    conda-forge
matplotlib-inline         0.1.6              pyhd8ed1ab_0    conda-forge
minikanren                1.0.3              pyhd8ed1ab_0    conda-forge
mkl                       2021.4.0           h06a4308_640  
multipledispatch          0.6.0                      py_0    conda-forge
munkres                   1.1.4              pyh9f0ad1d_0    conda-forge
ncurses                   6.4                  h59595ed_2    conda-forge
nest-asyncio              1.5.8              pyhd8ed1ab_0    conda-forge
numpy                     1.25.2          py311h64a7726_0    conda-forge
openjpeg                  2.5.0                h488ebb8_3    conda-forge
openssl                   3.1.4                hd590300_0    conda-forge
packaging                 23.2               pyhd8ed1ab_0    conda-forge
pandas                    2.1.2           py311h320fe9a_0    conda-forge
pango                     1.50.14              ha41ecd1_2    conda-forge
parso                     0.8.3              pyhd8ed1ab_0    conda-forge
pcre2                     10.40                hc3806b6_0    conda-forge
pexpect                   4.8.0              pyh1a96a4e_2    conda-forge
pickleshare               0.7.5                   py_1003    conda-forge
pillow                    10.1.0          py311ha6c5da5_0    conda-forge
pip                       23.3.1             pyhd8ed1ab_0    conda-forge
pixman                    0.42.2               h59595ed_0    conda-forge
platformdirs              3.11.0             pyhd8ed1ab_0    conda-forge
prompt-toolkit            3.0.39             pyha770c72_0    conda-forge
prompt_toolkit            3.0.39               hd8ed1ab_0    conda-forge
psutil                    5.9.5           py311h459d7ec_1    conda-forge
pthread-stubs             0.4               h36c2ea0_1001    conda-forge
ptyprocess                0.7.0              pyhd3deb0d_0    conda-forge
pure_eval                 0.2.2              pyhd8ed1ab_0    conda-forge
pygments                  2.16.1             pyhd8ed1ab_0    conda-forge
pymc                      5.9.1                hd8ed1ab_0    conda-forge
pymc-base                 5.9.1              pyhd8ed1ab_0    conda-forge
pyparsing                 3.1.1              pyhd8ed1ab_0    conda-forge
pytensor                  2.13.1          py311ha02d727_0  
pytensor-base             2.17.3          py311h320fe9a_0    conda-forge
python                    3.11.6          hab00c5b_0_cpython    conda-forge
python-dateutil           2.8.2              pyhd8ed1ab_0    conda-forge
python-graphviz           0.20.1             pyh22cad53_0    conda-forge
python-tzdata             2023.3             pyhd8ed1ab_0    conda-forge
python_abi                3.11                    4_cp311    conda-forge
pytz                      2023.3.post1       pyhd8ed1ab_0    conda-forge
pyzmq                     25.1.1          py311h34ded2d_2    conda-forge
readline                  8.2                  h8228510_1    conda-forge
scikit-learn              1.3.2           py311hc009520_1    conda-forge
scipy                     1.11.3          py311h64a7726_1    conda-forge
setuptools                68.2.2             pyhd8ed1ab_0    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
spyder-kernels            2.4.4           unix_pyh707e725_0    conda-forge
stack_data                0.6.2              pyhd8ed1ab_0    conda-forge
threadpoolctl             3.2.0              pyha21a80b_0    conda-forge
tk                        8.6.13               h2797004_0    conda-forge
toolz                     0.12.0             pyhd8ed1ab_0    conda-forge
tornado                   6.3.3           py311h459d7ec_1    conda-forge
traitlets                 5.13.0             pyhd8ed1ab_0    conda-forge
typing-extensions         4.8.0                hd8ed1ab_0    conda-forge
typing_extensions         4.8.0              pyha770c72_0    conda-forge
tzdata                    2023c                h71feb2d_0    conda-forge
wcwidth                   0.2.9              pyhd8ed1ab_0    conda-forge
wheel                     0.41.3             pyhd8ed1ab_0    conda-forge
wurlitzer                 3.0.3              pyhd8ed1ab_0    conda-forge
xarray                    2023.10.1          pyhd8ed1ab_0    conda-forge
xarray-einstats           0.6.0              pyhd8ed1ab_0    conda-forge
xorg-kbproto              1.0.7             h7f98852_1002    conda-forge
xorg-libice               1.1.1                hd590300_0    conda-forge
xorg-libsm                1.2.4                h7391055_0    conda-forge
xorg-libx11               1.8.7                h8ee46fc_0    conda-forge
xorg-libxau               1.0.11               hd590300_0    conda-forge
xorg-libxdmcp             1.1.3                h7f98852_0    conda-forge
xorg-libxext              1.3.4                h0b41bf4_2    conda-forge
xorg-libxrender           0.9.11               hd590300_0    conda-forge
xorg-renderproto          0.11.1            h7f98852_1002    conda-forge
xorg-xextproto            7.3.0             h0b41bf4_1003    conda-forge
xorg-xproto               7.0.31            h7f98852_1007    conda-forge
xz                        5.2.6                h166bdaf_0    conda-forge
zeromq                    4.3.5                h59595ed_0    conda-forge
zipp                      3.17.0             pyhd8ed1ab_0    conda-forge
zlib                      1.2.13               hd590300_5    conda-forge
zstd                      1.5.5                hfc55251_0    conda-forge

[jessegrabowski]

It still didn’t grab mkl-service, can you install that as well? You might also try to do a fresh env specifying all of pymc==5.9.1 libblas=*=*mkl mkl-service

@maresb any idea why mkl is being left out of @iavicenna’s 5.9.1 installation?

[iavicenna]

@jessegrabowski

libblas=*=*mkl mkl-service

Did it as you suggested now. Still extremely slow. Here is a list of packages

# packages in environment at /home/avicenna/miniconda3/envs/pymc_env_5_9_1_mkl:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                  2_kmp_llvm    conda-forge
arviz                     0.16.1             pyhd8ed1ab_1    conda-forge
asttokens                 2.4.1              pyhd8ed1ab_0    conda-forge
atk-1.0                   2.38.0               hd4edc92_1    conda-forge
backports                 1.0                pyhd8ed1ab_3    conda-forge
backports.functools_lru_cache 1.6.5              pyhd8ed1ab_0    conda-forge
brotli                    1.1.0                hd590300_1    conda-forge
brotli-bin                1.1.0                hd590300_1    conda-forge
bzip2                     1.0.8                h7f98852_4    conda-forge
c-ares                    1.20.1               hd590300_1    conda-forge
ca-certificates           2023.7.22            hbcca054_0    conda-forge
cached-property           1.5.2                hd8ed1ab_1    conda-forge
cached_property           1.5.2              pyha770c72_1    conda-forge
cachetools                5.3.2              pyhd8ed1ab_0    conda-forge
cairo                     1.18.0               h3faef2a_0    conda-forge
certifi                   2023.7.22          pyhd8ed1ab_0    conda-forge
cloudpickle               3.0.0              pyhd8ed1ab_0    conda-forge
comm                      0.1.4              pyhd8ed1ab_0    conda-forge
cons                      0.4.6              pyhd8ed1ab_0    conda-forge
contourpy                 1.1.1           py311h9547e67_1    conda-forge
cycler                    0.12.1             pyhd8ed1ab_0    conda-forge
debugpy                   1.8.0           py311hb755f60_1    conda-forge
decorator                 5.1.1              pyhd8ed1ab_0    conda-forge
etuples                   0.3.9              pyhd8ed1ab_0    conda-forge
exceptiongroup            1.1.3              pyhd8ed1ab_0    conda-forge
executing                 2.0.1              pyhd8ed1ab_0    conda-forge
expat                     2.5.0                hcb278e6_1    conda-forge
fastprogress              1.0.3              pyhd8ed1ab_0    conda-forge
filelock                  3.13.1             pyhd8ed1ab_0    conda-forge
font-ttf-dejavu-sans-mono 2.37                 hab24e00_0    conda-forge
font-ttf-inconsolata      3.000                h77eed37_0    conda-forge
font-ttf-source-code-pro  2.038                h77eed37_0    conda-forge
font-ttf-ubuntu           0.83                 hab24e00_0    conda-forge
fontconfig                2.14.2               h14ed4e7_0    conda-forge
fonts-conda-ecosystem     1                             0    conda-forge
fonts-conda-forge         1                             0    conda-forge
fonttools                 4.43.1          py311h459d7ec_0    conda-forge
freetype                  2.12.1               h267a509_2    conda-forge
fribidi                   1.0.10               h36c2ea0_0    conda-forge
gdk-pixbuf                2.42.10              h829c605_4    conda-forge
gettext                   0.21.1               h27087fc_0    conda-forge
giflib                    5.2.1                h0b41bf4_3    conda-forge
graphite2                 1.3.13            h58526e2_1001    conda-forge
graphviz                  8.1.0                h28d9a01_0    conda-forge
gtk2                      2.24.33              h90689f9_2    conda-forge
gts                       0.7.6                h977cf35_4    conda-forge
h5netcdf                  1.2.0              pyhd8ed1ab_0    conda-forge
h5py                      3.10.0          nompi_py311h3839ddf_100    conda-forge
harfbuzz                  8.2.1                h3d44ed6_0    conda-forge
hdf5                      1.14.2          nompi_h4f84152_100    conda-forge
icu                       73.2                 h59595ed_0    conda-forge
importlib-metadata        6.8.0              pyha770c72_0    conda-forge
importlib_metadata        6.8.0                hd8ed1ab_0    conda-forge
ipykernel                 6.26.0             pyhf8b6a83_0    conda-forge
ipython                   8.17.2             pyh41d4057_0    conda-forge
jedi                      0.19.1             pyhd8ed1ab_0    conda-forge
joblib                    1.3.2              pyhd8ed1ab_0    conda-forge
jupyter_client            8.5.0              pyhd8ed1ab_0    conda-forge
jupyter_core              5.5.0           py311h38be061_0    conda-forge
keyutils                  1.6.1                h166bdaf_0    conda-forge
kiwisolver                1.4.5           py311h9547e67_1    conda-forge
krb5                      1.21.2               h659d440_0    conda-forge
lcms2                     2.15                 hb7c19ff_3    conda-forge
ld_impl_linux-64          2.40                 h41732ed_0    conda-forge
lerc                      4.0.0                h27087fc_0    conda-forge
libaec                    1.1.2                h59595ed_1    conda-forge
libblas                   3.9.0            16_linux64_mkl    conda-forge
libbrotlicommon           1.1.0                hd590300_1    conda-forge
libbrotlidec              1.1.0                hd590300_1    conda-forge
libbrotlienc              1.1.0                hd590300_1    conda-forge
libcblas                  3.9.0            16_linux64_mkl    conda-forge
libcurl                   8.4.0                hca28451_0    conda-forge
libdeflate                1.19                 hd590300_0    conda-forge
libedit                   3.1.20191231         he28a2e2_2    conda-forge
libev                     4.33                 h516909a_1    conda-forge
libexpat                  2.5.0                hcb278e6_1    conda-forge
libffi                    3.4.2                h7f98852_5    conda-forge
libgcc-ng                 13.2.0               h807b86a_2    conda-forge
libgd                     2.3.3                h119a65a_9    conda-forge
libgfortran-ng            13.2.0               h69a702a_2    conda-forge
libgfortran5              13.2.0               ha4646dd_2    conda-forge
libglib                   2.78.0               hebfc3b9_0    conda-forge
libhwloc                  2.9.3           default_h554bfaf_1009    conda-forge
libiconv                  1.17                 h166bdaf_0    conda-forge
libjpeg-turbo             3.0.0                hd590300_1    conda-forge
liblapack                 3.9.0            16_linux64_mkl    conda-forge
libnghttp2                1.55.1               h47da74e_0    conda-forge
libnsl                    2.0.1                hd590300_0    conda-forge
libpng                    1.6.39               h753d276_0    conda-forge
librsvg                   2.56.3               h98fae49_0    conda-forge
libsodium                 1.0.18               h36c2ea0_1    conda-forge
libsqlite                 3.43.2               h2797004_0    conda-forge
libssh2                   1.11.0               h0841786_0    conda-forge
libstdcxx-ng              13.2.0               h7e041cc_2    conda-forge
libtiff                   4.6.0                ha9c0a0a_2    conda-forge
libtool                   2.4.7                h27087fc_0    conda-forge
libuuid                   2.38.1               h0b41bf4_0    conda-forge
libwebp                   1.3.2                h658648e_1    conda-forge
libwebp-base              1.3.2                hd590300_0    conda-forge
libxcb                    1.15                 h0b41bf4_0    conda-forge
libxml2                   2.11.5               h232c23b_1    conda-forge
libzlib                   1.2.13               hd590300_5    conda-forge
llvm-openmp               17.0.4               h4dfa4b3_0    conda-forge
logical-unification       0.4.6              pyhd8ed1ab_0    conda-forge
matplotlib-base           3.8.0           py311h54ef318_2    conda-forge
matplotlib-inline         0.1.6              pyhd8ed1ab_0    conda-forge
minikanren                1.0.3              pyhd8ed1ab_0    conda-forge
mkl                       2022.2.1         h84fe81f_16997    conda-forge
mkl-service               2.4.0           py311hb711fc7_0    conda-forge
multipledispatch          0.6.0                      py_0    conda-forge
munkres                   1.1.4              pyh9f0ad1d_0    conda-forge
ncurses                   6.4                  h59595ed_2    conda-forge
nest-asyncio              1.5.8              pyhd8ed1ab_0    conda-forge
numpy                     1.25.2          py311h64a7726_0    conda-forge
openjpeg                  2.5.0                h488ebb8_3    conda-forge
openssl                   3.1.4                hd590300_0    conda-forge
packaging                 23.2               pyhd8ed1ab_0    conda-forge
pandas                    2.1.2           py311h320fe9a_0    conda-forge
pango                     1.50.14              ha41ecd1_2    conda-forge
parso                     0.8.3              pyhd8ed1ab_0    conda-forge
pcre2                     10.40                hc3806b6_0    conda-forge
pexpect                   4.8.0              pyh1a96a4e_2    conda-forge
pickleshare               0.7.5                   py_1003    conda-forge
pillow                    10.1.0          py311ha6c5da5_0    conda-forge
pip                       23.3.1             pyhd8ed1ab_0    conda-forge
pixman                    0.42.2               h59595ed_0    conda-forge
platformdirs              3.11.0             pyhd8ed1ab_0    conda-forge
prompt-toolkit            3.0.39             pyha770c72_0    conda-forge
prompt_toolkit            3.0.39               hd8ed1ab_0    conda-forge
psutil                    5.9.5           py311h459d7ec_1    conda-forge
pthread-stubs             0.4               h36c2ea0_1001    conda-forge
ptyprocess                0.7.0              pyhd3deb0d_0    conda-forge
pure_eval                 0.2.2              pyhd8ed1ab_0    conda-forge
pygments                  2.16.1             pyhd8ed1ab_0    conda-forge
pymc                      5.9.1                hd8ed1ab_0    conda-forge
pymc-base                 5.9.1              pyhd8ed1ab_0    conda-forge
pyparsing                 3.1.1              pyhd8ed1ab_0    conda-forge
pytensor                  2.13.1          py311ha02d727_0  
pytensor-base             2.17.3          py311h320fe9a_0    conda-forge
python                    3.11.6          hab00c5b_0_cpython    conda-forge
python-dateutil           2.8.2              pyhd8ed1ab_0    conda-forge
python-graphviz           0.20.1             pyh22cad53_0    conda-forge
python-tzdata             2023.3             pyhd8ed1ab_0    conda-forge
python_abi                3.11                    4_cp311    conda-forge
pytz                      2023.3.post1       pyhd8ed1ab_0    conda-forge
pyzmq                     25.1.1          py311h34ded2d_2    conda-forge
readline                  8.2                  h8228510_1    conda-forge
scikit-learn              1.3.2           py311hc009520_1    conda-forge
scipy                     1.11.3          py311h64a7726_1    conda-forge
setuptools                68.2.2             pyhd8ed1ab_0    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
spyder-kernels            2.4.4           unix_pyh707e725_0    conda-forge
stack_data                0.6.2              pyhd8ed1ab_0    conda-forge
tbb                       2021.10.0            h00ab1b0_2    conda-forge
threadpoolctl             3.2.0              pyha21a80b_0    conda-forge
tk                        8.6.13               h2797004_0    conda-forge
toolz                     0.12.0             pyhd8ed1ab_0    conda-forge
tornado                   6.3.3           py311h459d7ec_1    conda-forge
traitlets                 5.13.0             pyhd8ed1ab_0    conda-forge
typing-extensions         4.8.0                hd8ed1ab_0    conda-forge
typing_extensions         4.8.0              pyha770c72_0    conda-forge
tzdata                    2023c                h71feb2d_0    conda-forge
wcwidth                   0.2.9              pyhd8ed1ab_0    conda-forge
wheel                     0.41.3             pyhd8ed1ab_0    conda-forge
wurlitzer                 3.0.3              pyhd8ed1ab_0    conda-forge
xarray                    2023.10.1          pyhd8ed1ab_0    conda-forge
xarray-einstats           0.6.0              pyhd8ed1ab_0    conda-forge
xorg-kbproto              1.0.7             h7f98852_1002    conda-forge
xorg-libice               1.1.1                hd590300_0    conda-forge
xorg-libsm                1.2.4                h7391055_0    conda-forge
xorg-libx11               1.8.7                h8ee46fc_0    conda-forge
xorg-libxau               1.0.11               hd590300_0    conda-forge
xorg-libxdmcp             1.1.3                h7f98852_0    conda-forge
xorg-libxext              1.3.4                h0b41bf4_2    conda-forge
xorg-libxrender           0.9.11               hd590300_0    conda-forge
xorg-renderproto          0.11.1            h7f98852_1002    conda-forge
xorg-xextproto            7.3.0             h0b41bf4_1003    conda-forge
xorg-xproto               7.0.31            h7f98852_1007    conda-forge
xz                        5.2.6                h166bdaf_0    conda-forge
zeromq                    4.3.5                h59595ed_0    conda-forge
zipp                      3.17.0             pyhd8ed1ab_0    conda-forge
zlib                      1.2.13               hd590300_5    conda-forge
zstd                      1.5.5                hfc55251_0    conda-forge

Comparing to 5.9 which is the closest version that works (albeit being slower than 5.8):

backcall not found
binutils_impl_linux-64 not found
binutils_linux-64 not found
blas not found
blas-devel not found
gcc not found
gcc_impl_linux-64 not found
gcc_linux-64 not found
gxx not found
gxx_impl_linux-64 not found
gxx_linux-64 not found
ipython found but mismatch. Specification pkg: ipython==8.16.1=pyh0d859eb_0, Running pkg: ipython==8.17.2=pyh41d4057_0
kernel-headers_linux-64 not found
libblas found but mismatch. Specification pkg: libblas==3.9.0=19_linux64_openblas, Running pkg: libblas==3.9.0=16_linux64_mkl
libcblas found but mismatch. Specification pkg: libcblas==3.9.0=19_linux64_openblas, Running pkg: libcblas==3.9.0=16_linux64_mkl
libgcc-devel_linux-64 not found
libgomp not found
liblapack found but mismatch. Specification pkg: liblapack==3.9.0=19_linux64_openblas, Running pkg: liblapack==3.9.0=16_linux64_mkl
liblapacke not found
libopenblas not found
libsanitizer not found
libstdcxx-devel_linux-64 not found
llvm-openmp found but mismatch. Specification pkg: llvm-openmp==17.0.3=h4dfa4b3_0, Running pkg: llvm-openmp==17.0.4=h4dfa4b3_0
openblas not found
pymc found but mismatch. Specification pkg: pymc==5.9.0=hd8ed1ab_0, Running pkg: pymc==5.9.1=hd8ed1ab_0
pymc-base found but mismatch. Specification pkg: pymc-base==5.9.0=pyhd8ed1ab_0, Running pkg: pymc-base==5.9.1=pyhd8ed1ab_0
pytensor found but mismatch. Specification pkg: pytensor==2.17.3=py311hb755f60_0, Running pkg: pytensor==2.13.1=py311ha02d727_0
sysroot_linux-64 not found

[iavicenna]

I dont know if it helps but I took the 5.9.1 pymc environment, uninstalled 5.9.1 and installed 5.8 and it again worked fine. While doing this here is what changed (when uninstalling it upgrades numpy from 1.25 to 1.26 and then when reinstalling it downgrades, in case you are wondering about that change):

UNINSTALL PYMC 5.9.1

The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    llvm-openmp-17.0.4         |       h4dfa4b3_0        39.4 MB  conda-forge
    numpy-1.26.0               |  py311h64a7726_0         7.7 MB  conda-forge
    ------------------------------------------------------------
                                           Total:        47.0 MB

The following NEW packages will be INSTALLED:

  libhwloc           conda-forge/linux-64::libhwloc-2.9.3-default_h554bfaf_1009 
  llvm-openmp        conda-forge/linux-64::llvm-openmp-17.0.4-h4dfa4b3_0 
  tbb                conda-forge/linux-64::tbb-2021.10.0-h00ab1b0_2 

The following packages will be REMOVED:

  arviz-0.16.1-pyhd8ed1ab_1
  atk-1.0-2.38.0-hd4edc92_1
  brotli-1.1.0-hd590300_1
  brotli-bin-1.1.0-hd590300_1
  c-ares-1.20.1-hd590300_1
  cached-property-1.5.2-hd8ed1ab_1
  cached_property-1.5.2-pyha770c72_1
  cachetools-5.3.2-pyhd8ed1ab_0
  cairo-1.18.0-h3faef2a_0
  cons-0.4.6-pyhd8ed1ab_0
  contourpy-1.1.1-py311h9547e67_1
  cycler-0.12.1-pyhd8ed1ab_0
  etuples-0.3.9-pyhd8ed1ab_0
  expat-2.5.0-hcb278e6_1
  fastprogress-1.0.3-pyhd8ed1ab_0
  filelock-3.13.1-pyhd8ed1ab_0
  font-ttf-dejavu-sans-mono-2.37-hab24e00_0
  font-ttf-inconsolata-3.000-h77eed37_0
  font-ttf-source-code-pro-2.038-h77eed37_0
  font-ttf-ubuntu-0.83-hab24e00_0
  fontconfig-2.14.2-h14ed4e7_0
  fonts-conda-ecosystem-1-0
  fonts-conda-forge-1-0
  fonttools-4.43.1-py311h459d7ec_0
  freetype-2.12.1-h267a509_2
  fribidi-1.0.10-h36c2ea0_0
  gdk-pixbuf-2.42.10-h829c605_4
  gettext-0.21.1-h27087fc_0
  giflib-5.2.1-h0b41bf4_3
  graphite2-1.3.13-h58526e2_1001
  graphviz-8.1.0-h28d9a01_0
  gtk2-2.24.33-h90689f9_2
  gts-0.7.6-h977cf35_4
  h5netcdf-1.2.0-pyhd8ed1ab_0
  h5py-3.10.0-nompi_py311h3839ddf_100
  harfbuzz-8.2.1-h3d44ed6_0
  hdf5-1.14.2-nompi_h4f84152_100
  intel-openmp-2021.4.0-h06a4308_3561
  keyutils-1.6.1-h166bdaf_0
  kiwisolver-1.4.5-py311h9547e67_1
  krb5-1.21.2-h659d440_0
  lcms2-2.15-hb7c19ff_3
  lerc-4.0.0-h27087fc_0
  libaec-1.1.2-h59595ed_1
  libbrotlicommon-1.1.0-hd590300_1
  libbrotlidec-1.1.0-hd590300_1
  libbrotlienc-1.1.0-hd590300_1
  libcurl-8.4.0-hca28451_0
  libdeflate-1.19-hd590300_0
  libedit-3.1.20191231-he28a2e2_2
  libev-4.33-h516909a_1
  libgd-2.3.3-h119a65a_9
  libglib-2.78.0-hebfc3b9_0
  libgomp-13.2.0-h807b86a_2
  libjpeg-turbo-3.0.0-hd590300_1
  libnghttp2-1.55.1-h47da74e_0
  libopenblas-0.3.24-pthreads_h413a1c8_0
  libpng-1.6.39-h753d276_0
  librsvg-2.56.3-h98fae49_0
  libssh2-1.11.0-h0841786_0
  libtiff-4.6.0-ha9c0a0a_2
  libtool-2.4.7-h27087fc_0
  libwebp-1.3.2-h658648e_1
  libwebp-base-1.3.2-hd590300_0
  libxcb-1.15-h0b41bf4_0
  logical-unification-0.4.6-pyhd8ed1ab_0
  matplotlib-base-3.8.0-py311h54ef318_2
  minikanren-1.0.3-pyhd8ed1ab_0
  multipledispatch-0.6.0-py_0
  munkres-1.1.4-pyh9f0ad1d_0
  openjpeg-2.5.0-h488ebb8_3
  pandas-2.1.2-py311h320fe9a_0
  pango-1.50.14-ha41ecd1_2
  pcre2-10.40-hc3806b6_0
  pillow-10.1.0-py311ha6c5da5_0
  pixman-0.42.2-h59595ed_0
  pthread-stubs-0.4-h36c2ea0_1001
  pymc-5.9.1-hd8ed1ab_0
  pymc-base-5.9.1-pyhd8ed1ab_0
  pyparsing-3.1.1-pyhd8ed1ab_0
  pytensor-2.13.1-py311ha02d727_0
  pytensor-base-2.17.3-py311h320fe9a_0
  python-graphviz-0.20.1-pyh22cad53_0
  python-tzdata-2023.3-pyhd8ed1ab_0
  pytz-2023.3.post1-pyhd8ed1ab_0
  toolz-0.12.0-pyhd8ed1ab_0
  xarray-2023.10.1-pyhd8ed1ab_0
  xarray-einstats-0.6.0-pyhd8ed1ab_0
  xorg-kbproto-1.0.7-h7f98852_1002
  xorg-libice-1.1.1-hd590300_0
  xorg-libsm-1.2.4-h7391055_0
  xorg-libx11-1.8.7-h8ee46fc_0
  xorg-libxau-1.0.11-hd590300_0
  xorg-libxdmcp-1.1.3-h7f98852_0
  xorg-libxext-1.3.4-h0b41bf4_2
  xorg-libxrender-0.9.11-hd590300_0
  xorg-renderproto-0.11.1-h7f98852_1002
  xorg-xextproto-7.3.0-h0b41bf4_1003
  xorg-xproto-7.0.31-h7f98852_1007
  zlib-1.2.13-hd590300_5

The following packages will be UPDATED:

  mkl                  pkgs/main::mkl-2021.4.0-h06a4308_640 --> conda-forge::mkl-2021.4.0-h8d4b97c_729 
  numpy                              1.25.2-py311h64a7726_0 --> 1.26.0-py311h64a7726_0 

The following packages will be DOWNGRADED:

  _openmp_mutex                                   4.5-2_gnu --> 4.5-2_kmp_llvm 
  
  
INSTALL PYMC 5.8
  
 ## Package Plan ##

  environment location: /home/avicenna/miniconda3/envs/pymc_env_5_9_1

  added / updated specs:
    - pymc=5.8


The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    pymc-5.8.2                 |       hd8ed1ab_0          11 KB  conda-forge
    pymc-base-5.8.2            |     pyhd8ed1ab_0         308 KB  conda-forge
    pytensor-base-2.16.2       |  py311h320fe9a_0         2.1 MB  conda-forge
    ------------------------------------------------------------
                                           Total:         2.4 MB

The following NEW packages will be INSTALLED:

  arviz              conda-forge/noarch::arviz-0.16.1-pyhd8ed1ab_1 
  atk-1.0            conda-forge/linux-64::atk-1.0-2.38.0-hd4edc92_1 
  brotli             conda-forge/linux-64::brotli-1.1.0-hd590300_1 
  brotli-bin         conda-forge/linux-64::brotli-bin-1.1.0-hd590300_1 
  c-ares             conda-forge/linux-64::c-ares-1.20.1-hd590300_1 
  cached-property    conda-forge/noarch::cached-property-1.5.2-hd8ed1ab_1 
  cached_property    conda-forge/noarch::cached_property-1.5.2-pyha770c72_1 
  cachetools         conda-forge/noarch::cachetools-5.3.2-pyhd8ed1ab_0 
  cairo              conda-forge/linux-64::cairo-1.18.0-h3faef2a_0 
  cons               conda-forge/noarch::cons-0.4.6-pyhd8ed1ab_0 
  contourpy          conda-forge/linux-64::contourpy-1.1.1-py311h9547e67_1 
  cycler             conda-forge/noarch::cycler-0.12.1-pyhd8ed1ab_0 
  etuples            conda-forge/noarch::etuples-0.3.9-pyhd8ed1ab_0 
  expat              conda-forge/linux-64::expat-2.5.0-hcb278e6_1 
  fastprogress       conda-forge/noarch::fastprogress-1.0.3-pyhd8ed1ab_0 
  filelock           conda-forge/noarch::filelock-3.13.1-pyhd8ed1ab_0 
  font-ttf-dejavu-s~ conda-forge/noarch::font-ttf-dejavu-sans-mono-2.37-hab24e00_0 
  font-ttf-inconsol~ conda-forge/noarch::font-ttf-inconsolata-3.000-h77eed37_0 
  font-ttf-source-c~ conda-forge/noarch::font-ttf-source-code-pro-2.038-h77eed37_0 
  font-ttf-ubuntu    conda-forge/noarch::font-ttf-ubuntu-0.83-hab24e00_0 
  fontconfig         conda-forge/linux-64::fontconfig-2.14.2-h14ed4e7_0 
  fonts-conda-ecosy~ conda-forge/noarch::fonts-conda-ecosystem-1-0 
  fonts-conda-forge  conda-forge/noarch::fonts-conda-forge-1-0 
  fonttools          conda-forge/linux-64::fonttools-4.43.1-py311h459d7ec_0 
  freetype           conda-forge/linux-64::freetype-2.12.1-h267a509_2 
  fribidi            conda-forge/linux-64::fribidi-1.0.10-h36c2ea0_0 
  gdk-pixbuf         conda-forge/linux-64::gdk-pixbuf-2.42.10-h829c605_4 
  gettext            conda-forge/linux-64::gettext-0.21.1-h27087fc_0 
  giflib             conda-forge/linux-64::giflib-5.2.1-h0b41bf4_3 
  graphite2          conda-forge/linux-64::graphite2-1.3.13-h58526e2_1001 
  graphviz           conda-forge/linux-64::graphviz-8.1.0-h28d9a01_0 
  gtk2               conda-forge/linux-64::gtk2-2.24.33-h90689f9_2 
  gts                conda-forge/linux-64::gts-0.7.6-h977cf35_4 
  h5netcdf           conda-forge/noarch::h5netcdf-1.2.0-pyhd8ed1ab_0 
  h5py               conda-forge/linux-64::h5py-3.10.0-nompi_py311h3839ddf_100 
  harfbuzz           conda-forge/linux-64::harfbuzz-8.2.1-h3d44ed6_0 
  hdf5               conda-forge/linux-64::hdf5-1.14.2-nompi_h4f84152_100 
  keyutils           conda-forge/linux-64::keyutils-1.6.1-h166bdaf_0 
  kiwisolver         conda-forge/linux-64::kiwisolver-1.4.5-py311h9547e67_1 
  krb5               conda-forge/linux-64::krb5-1.21.2-h659d440_0 
  lcms2              conda-forge/linux-64::lcms2-2.15-hb7c19ff_3 
  lerc               conda-forge/linux-64::lerc-4.0.0-h27087fc_0 
  libaec             conda-forge/linux-64::libaec-1.1.2-h59595ed_1 
  libbrotlicommon    conda-forge/linux-64::libbrotlicommon-1.1.0-hd590300_1 
  libbrotlidec       conda-forge/linux-64::libbrotlidec-1.1.0-hd590300_1 
  libbrotlienc       conda-forge/linux-64::libbrotlienc-1.1.0-hd590300_1 
  libcurl            conda-forge/linux-64::libcurl-8.4.0-hca28451_0 
  libdeflate         conda-forge/linux-64::libdeflate-1.19-hd590300_0 
  libedit            conda-forge/linux-64::libedit-3.1.20191231-he28a2e2_2 
  libev              conda-forge/linux-64::libev-4.33-h516909a_1 
  libgd              conda-forge/linux-64::libgd-2.3.3-h119a65a_9 
  libglib            conda-forge/linux-64::libglib-2.78.0-hebfc3b9_0 
  libjpeg-turbo      conda-forge/linux-64::libjpeg-turbo-3.0.0-hd590300_1 
  libnghttp2         conda-forge/linux-64::libnghttp2-1.55.1-h47da74e_0 
  libpng             conda-forge/linux-64::libpng-1.6.39-h753d276_0 
  librsvg            conda-forge/linux-64::librsvg-2.56.3-h98fae49_0 
  libssh2            conda-forge/linux-64::libssh2-1.11.0-h0841786_0 
  libtiff            conda-forge/linux-64::libtiff-4.6.0-ha9c0a0a_2 
  libtool            conda-forge/linux-64::libtool-2.4.7-h27087fc_0 
  libwebp            conda-forge/linux-64::libwebp-1.3.2-h658648e_1 
  libwebp-base       conda-forge/linux-64::libwebp-base-1.3.2-hd590300_0 
  libxcb             conda-forge/linux-64::libxcb-1.15-h0b41bf4_0 
  logical-unificati~ conda-forge/noarch::logical-unification-0.4.6-pyhd8ed1ab_0 
  matplotlib-base    conda-forge/linux-64::matplotlib-base-3.8.0-py311h54ef318_2 
  minikanren         conda-forge/noarch::minikanren-1.0.3-pyhd8ed1ab_0 
  multipledispatch   conda-forge/noarch::multipledispatch-0.6.0-py_0 
  munkres            conda-forge/noarch::munkres-1.1.4-pyh9f0ad1d_0 
  openjpeg           conda-forge/linux-64::openjpeg-2.5.0-h488ebb8_3 
  pandas             conda-forge/linux-64::pandas-2.1.2-py311h320fe9a_0 
  pango              conda-forge/linux-64::pango-1.50.14-ha41ecd1_2 
  pcre2              conda-forge/linux-64::pcre2-10.40-hc3806b6_0 
  pillow             conda-forge/linux-64::pillow-10.1.0-py311ha6c5da5_0 
  pixman             conda-forge/linux-64::pixman-0.42.2-h59595ed_0 
  pthread-stubs      conda-forge/linux-64::pthread-stubs-0.4-h36c2ea0_1001 
  pymc               conda-forge/noarch::pymc-5.8.2-hd8ed1ab_0 
  pymc-base          conda-forge/noarch::pymc-base-5.8.2-pyhd8ed1ab_0 
  pyparsing          conda-forge/noarch::pyparsing-3.1.1-pyhd8ed1ab_0 
  pytensor           pkgs/main/linux-64::pytensor-2.13.1-py311ha02d727_0 
  pytensor-base      conda-forge/linux-64::pytensor-base-2.16.2-py311h320fe9a_0 
  python-graphviz    conda-forge/noarch::python-graphviz-0.20.1-pyh22cad53_0 
  python-tzdata      conda-forge/noarch::python-tzdata-2023.3-pyhd8ed1ab_0 
  pytz               conda-forge/noarch::pytz-2023.3.post1-pyhd8ed1ab_0 
  toolz              conda-forge/noarch::toolz-0.12.0-pyhd8ed1ab_0 
  xarray             conda-forge/noarch::xarray-2023.10.1-pyhd8ed1ab_0 
  xarray-einstats    conda-forge/noarch::xarray-einstats-0.6.0-pyhd8ed1ab_0 
  xorg-kbproto       conda-forge/linux-64::xorg-kbproto-1.0.7-h7f98852_1002 
  xorg-libice        conda-forge/linux-64::xorg-libice-1.1.1-hd590300_0 
  xorg-libsm         conda-forge/linux-64::xorg-libsm-1.2.4-h7391055_0 
  xorg-libx11        conda-forge/linux-64::xorg-libx11-1.8.7-h8ee46fc_0 
  xorg-libxau        conda-forge/linux-64::xorg-libxau-1.0.11-hd590300_0 
  xorg-libxdmcp      conda-forge/linux-64::xorg-libxdmcp-1.1.3-h7f98852_0 
  xorg-libxext       conda-forge/linux-64::xorg-libxext-1.3.4-h0b41bf4_2 
  xorg-libxrender    conda-forge/linux-64::xorg-libxrender-0.9.11-hd590300_0 
  xorg-renderproto   conda-forge/linux-64::xorg-renderproto-0.11.1-h7f98852_1002 
  xorg-xextproto     conda-forge/linux-64::xorg-xextproto-7.3.0-h0b41bf4_1003 
  xorg-xproto        conda-forge/linux-64::xorg-xproto-7.0.31-h7f98852_1007 
  zlib               conda-forge/linux-64::zlib-1.2.13-hd590300_5 

The following packages will be DOWNGRADED:

  numpy                              1.26.0-py311h64a7726_0 --> 1.25.2-py311h64a7726_0 

Listing the packages in this weird env where there speed-wise it is ok, I dont see mkl-service (though I see mkl)

_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                  2_kmp_llvm    conda-forge
arviz                     0.16.1             pyhd8ed1ab_1    conda-forge
asttokens                 2.4.1              pyhd8ed1ab_0    conda-forge
atk-1.0                   2.38.0               hd4edc92_1    conda-forge
backcall                  0.2.0              pyh9f0ad1d_0    conda-forge
backports                 1.0                pyhd8ed1ab_3    conda-forge
backports.functools_lru_cache 1.6.5              pyhd8ed1ab_0    conda-forge
brotli                    1.1.0                hd590300_1    conda-forge
brotli-bin                1.1.0                hd590300_1    conda-forge
bzip2                     1.0.8                h7f98852_4    conda-forge
c-ares                    1.20.1               hd590300_1    conda-forge
ca-certificates           2023.7.22            hbcca054_0    conda-forge
cached-property           1.5.2                hd8ed1ab_1    conda-forge
cached_property           1.5.2              pyha770c72_1    conda-forge
cachetools                5.3.2              pyhd8ed1ab_0    conda-forge
cairo                     1.18.0               h3faef2a_0    conda-forge
certifi                   2023.7.22          pyhd8ed1ab_0    conda-forge
cloudpickle               3.0.0              pyhd8ed1ab_0    conda-forge
comm                      0.1.4              pyhd8ed1ab_0    conda-forge
cons                      0.4.6              pyhd8ed1ab_0    conda-forge
contourpy                 1.1.1           py311h9547e67_1    conda-forge
cycler                    0.12.1             pyhd8ed1ab_0    conda-forge
debugpy                   1.8.0           py311hb755f60_1    conda-forge
decorator                 5.1.1              pyhd8ed1ab_0    conda-forge
etuples                   0.3.9              pyhd8ed1ab_0    conda-forge
exceptiongroup            1.1.3              pyhd8ed1ab_0    conda-forge
executing                 2.0.1              pyhd8ed1ab_0    conda-forge
expat                     2.5.0                hcb278e6_1    conda-forge
fastprogress              1.0.3              pyhd8ed1ab_0    conda-forge
filelock                  3.13.1             pyhd8ed1ab_0    conda-forge
font-ttf-dejavu-sans-mono 2.37                 hab24e00_0    conda-forge
font-ttf-inconsolata      3.000                h77eed37_0    conda-forge
font-ttf-source-code-pro  2.038                h77eed37_0    conda-forge
font-ttf-ubuntu           0.83                 hab24e00_0    conda-forge
fontconfig                2.14.2               h14ed4e7_0    conda-forge
fonts-conda-ecosystem     1                             0    conda-forge
fonts-conda-forge         1                             0    conda-forge
fonttools                 4.43.1          py311h459d7ec_0    conda-forge
freetype                  2.12.1               h267a509_2    conda-forge
fribidi                   1.0.10               h36c2ea0_0    conda-forge
gdk-pixbuf                2.42.10              h829c605_4    conda-forge
gettext                   0.21.1               h27087fc_0    conda-forge
giflib                    5.2.1                h0b41bf4_3    conda-forge
graphite2                 1.3.13            h58526e2_1001    conda-forge
graphviz                  8.1.0                h28d9a01_0    conda-forge
gtk2                      2.24.33              h90689f9_2    conda-forge
gts                       0.7.6                h977cf35_4    conda-forge
h5netcdf                  1.2.0              pyhd8ed1ab_0    conda-forge
h5py                      3.10.0          nompi_py311h3839ddf_100    conda-forge
harfbuzz                  8.2.1                h3d44ed6_0    conda-forge
hdf5                      1.14.2          nompi_h4f84152_100    conda-forge
icu                       73.2                 h59595ed_0    conda-forge
importlib-metadata        6.8.0              pyha770c72_0    conda-forge
importlib_metadata        6.8.0                hd8ed1ab_0    conda-forge
ipykernel                 6.26.0             pyhf8b6a83_0    conda-forge
ipython                   8.16.1             pyh0d859eb_0    conda-forge
jedi                      0.19.1             pyhd8ed1ab_0    conda-forge
joblib                    1.3.2              pyhd8ed1ab_0    conda-forge
jupyter_client            8.5.0              pyhd8ed1ab_0    conda-forge
jupyter_core              5.5.0           py311h38be061_0    conda-forge
keyutils                  1.6.1                h166bdaf_0    conda-forge
kiwisolver                1.4.5           py311h9547e67_1    conda-forge
krb5                      1.21.2               h659d440_0    conda-forge
lcms2                     2.15                 hb7c19ff_3    conda-forge
ld_impl_linux-64          2.40                 h41732ed_0    conda-forge
lerc                      4.0.0                h27087fc_0    conda-forge
libaec                    1.1.2                h59595ed_1    conda-forge
libblas                   3.9.0            12_linux64_mkl    conda-forge
libbrotlicommon           1.1.0                hd590300_1    conda-forge
libbrotlidec              1.1.0                hd590300_1    conda-forge
libbrotlienc              1.1.0                hd590300_1    conda-forge
libcblas                  3.9.0            12_linux64_mkl    conda-forge
libcurl                   8.4.0                hca28451_0    conda-forge
libdeflate                1.19                 hd590300_0    conda-forge
libedit                   3.1.20191231         he28a2e2_2    conda-forge
libev                     4.33                 h516909a_1    conda-forge
libexpat                  2.5.0                hcb278e6_1    conda-forge
libffi                    3.4.2                h7f98852_5    conda-forge
libgcc-ng                 13.2.0               h807b86a_2    conda-forge
libgd                     2.3.3                h119a65a_9    conda-forge
libgfortran-ng            13.2.0               h69a702a_2    conda-forge
libgfortran5              13.2.0               ha4646dd_2    conda-forge
libglib                   2.78.0               hebfc3b9_0    conda-forge
libhwloc                  2.9.3           default_h554bfaf_1009    conda-forge
libiconv                  1.17                 h166bdaf_0    conda-forge
libjpeg-turbo             3.0.0                hd590300_1    conda-forge
liblapack                 3.9.0            12_linux64_mkl    conda-forge
libnghttp2                1.55.1               h47da74e_0    conda-forge
libnsl                    2.0.1                hd590300_0    conda-forge
libpng                    1.6.39               h753d276_0    conda-forge
librsvg                   2.56.3               h98fae49_0    conda-forge
libsodium                 1.0.18               h36c2ea0_1    conda-forge
libsqlite                 3.43.2               h2797004_0    conda-forge
libssh2                   1.11.0               h0841786_0    conda-forge
libstdcxx-ng              13.2.0               h7e041cc_2    conda-forge
libtiff                   4.6.0                ha9c0a0a_2    conda-forge
libtool                   2.4.7                h27087fc_0    conda-forge
libuuid                   2.38.1               h0b41bf4_0    conda-forge
libwebp                   1.3.2                h658648e_1    conda-forge
libwebp-base              1.3.2                hd590300_0    conda-forge
libxcb                    1.15                 h0b41bf4_0    conda-forge
libxml2                   2.11.5               h232c23b_1    conda-forge
libzlib                   1.2.13               hd590300_5    conda-forge
llvm-openmp               17.0.4               h4dfa4b3_0    conda-forge
logical-unification       0.4.6              pyhd8ed1ab_0    conda-forge
matplotlib-base           3.8.0           py311h54ef318_2    conda-forge
matplotlib-inline         0.1.6              pyhd8ed1ab_0    conda-forge
minikanren                1.0.3              pyhd8ed1ab_0    conda-forge
mkl                       2021.4.0           h8d4b97c_729    conda-forge
multipledispatch          0.6.0                      py_0    conda-forge
munkres                   1.1.4              pyh9f0ad1d_0    conda-forge
ncurses                   6.4                  h59595ed_2    conda-forge
nest-asyncio              1.5.8              pyhd8ed1ab_0    conda-forge
numpy                     1.25.2          py311h64a7726_0    conda-forge
openjpeg                  2.5.0                h488ebb8_3    conda-forge
openssl                   3.1.4                hd590300_0    conda-forge
packaging                 23.2               pyhd8ed1ab_0    conda-forge
pandas                    2.1.2           py311h320fe9a_0    conda-forge
pango                     1.50.14              ha41ecd1_2    conda-forge
parso                     0.8.3              pyhd8ed1ab_0    conda-forge
pcre2                     10.40                hc3806b6_0    conda-forge
pexpect                   4.8.0              pyh1a96a4e_2    conda-forge
pickleshare               0.7.5                   py_1003    conda-forge
pillow                    10.1.0          py311ha6c5da5_0    conda-forge
pip                       23.3.1             pyhd8ed1ab_0    conda-forge
pixman                    0.42.2               h59595ed_0    conda-forge
platformdirs              3.11.0             pyhd8ed1ab_0    conda-forge
prompt-toolkit            3.0.39             pyha770c72_0    conda-forge
prompt_toolkit            3.0.39               hd8ed1ab_0    conda-forge
psutil                    5.9.5           py311h459d7ec_1    conda-forge
pthread-stubs             0.4               h36c2ea0_1001    conda-forge
ptyprocess                0.7.0              pyhd3deb0d_0    conda-forge
pure_eval                 0.2.2              pyhd8ed1ab_0    conda-forge
pygments                  2.16.1             pyhd8ed1ab_0    conda-forge
pymc                      5.8.2                hd8ed1ab_0    conda-forge
pymc-base                 5.8.2              pyhd8ed1ab_0    conda-forge
pyparsing                 3.1.1              pyhd8ed1ab_0    conda-forge
pytensor                  2.13.1          py311ha02d727_0  
pytensor-base             2.16.2          py311h320fe9a_0    conda-forge
python                    3.11.6          hab00c5b_0_cpython    conda-forge
python-dateutil           2.8.2              pyhd8ed1ab_0    conda-forge
python-graphviz           0.20.1             pyh22cad53_0    conda-forge
python-tzdata             2023.3             pyhd8ed1ab_0    conda-forge
python_abi                3.11                    4_cp311    conda-forge
pytz                      2023.3.post1       pyhd8ed1ab_0    conda-forge
pyzmq                     25.1.1          py311h34ded2d_2    conda-forge
readline                  8.2                  h8228510_1    conda-forge
scikit-learn              1.3.2           py311hc009520_1    conda-forge
scipy                     1.11.3          py311h64a7726_1    conda-forge
setuptools                68.2.2             pyhd8ed1ab_0    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
spyder-kernels            2.4.4           unix_pyh707e725_0    conda-forge
stack_data                0.6.2              pyhd8ed1ab_0    conda-forge
tbb                       2021.10.0            h00ab1b0_2    conda-forge
threadpoolctl             3.2.0              pyha21a80b_0    conda-forge
tk                        8.6.13               h2797004_0    conda-forge
toolz                     0.12.0             pyhd8ed1ab_0    conda-forge
tornado                   6.3.3           py311h459d7ec_1    conda-forge
traitlets                 5.13.0             pyhd8ed1ab_0    conda-forge
typing-extensions         4.8.0                hd8ed1ab_0    conda-forge
typing_extensions         4.8.0              pyha770c72_0    conda-forge
tzdata                    2023c                h71feb2d_0    conda-forge
wcwidth                   0.2.9              pyhd8ed1ab_0    conda-forge
wheel                     0.41.3             pyhd8ed1ab_0    conda-forge
wurlitzer                 3.0.3              pyhd8ed1ab_0    conda-forge
xarray                    2023.10.1          pyhd8ed1ab_0    conda-forge
xarray-einstats           0.6.0              pyhd8ed1ab_0    conda-forge
xorg-kbproto              1.0.7             h7f98852_1002    conda-forge
xorg-libice               1.1.1                hd590300_0    conda-forge
xorg-libsm                1.2.4                h7391055_0    conda-forge
xorg-libx11               1.8.7                h8ee46fc_0    conda-forge
xorg-libxau               1.0.11               hd590300_0    conda-forge
xorg-libxdmcp             1.1.3                h7f98852_0    conda-forge
xorg-libxext              1.3.4                h0b41bf4_2    conda-forge
xorg-libxrender           0.9.11               hd590300_0    conda-forge
xorg-renderproto          0.11.1            h7f98852_1002    conda-forge
xorg-xextproto            7.3.0             h0b41bf4_1003    conda-forge
xorg-xproto               7.0.31            h7f98852_1007    conda-forge
xz                        5.2.6                h166bdaf_0    conda-forge
zeromq                    4.3.5                h59595ed_0    conda-forge
zipp                      3.17.0             pyhd8ed1ab_0    conda-forge
zlib                      1.2.13               hd590300_5    conda-forge
zstd                      1.5.5                hfc55251_0    conda-forge

[iavicenna]

I have done one more test which seems to suggest that the extreme slow-down from 5.9 to 5.9.1 might be related to changes in pymc. I have installed a new env as follows:

conda create -n pymc_env_5_9_1_test2 pymc=5.9.1 scikit-learn spyder-kernels=2.4 _openmp_mutex==4.5=2_kmp_llvm llvm-openmp==17.0.3=h4dfa4b3_0 pytensor==2.17.3=py311hb755f60_0

These were most of the possibly related packages that differed between the 5.9 env and 5.9.1 env.
After installing it this way and comparing environments I get:

backcall not found
ipython found but mismatch. Specification pkg: ipython==8.16.1=pyh0d859eb_0, Running pkg: ipython==8.17.2=pyh41d4057_0
pymc found but mismatch. Specification pkg: pymc==5.9.0=hd8ed1ab_0, Running pkg: pymc==5.9.1=hd8ed1ab_0
pymc-base found but mismatch. Specification pkg: pymc-base==5.9.0=pyhd8ed1ab_0, Running pkg: pymc-base==5.9.1=pyhd8ed1ab_0

I ran the code in this env, dead slow. I re-ran the code in 5.9 (just to double check again), as before, somewhat slower than 5.8 but doable. Possibly not related but nevertheless I am going to try to create a final environment where backcall and ipython is also the same just to make sure.

ps: Ok that is also done and no improvement on speed. Here is the comparison between two environments

(pymc_env_5_9_1_test4) avicenna@avicenna:~/Desktop$ conda compare 5_9.yml
pymc found but mismatch. Specification pkg: pymc==5.9.0=hd8ed1ab_0, Running pkg: pymc==5.9.1=hd8ed1ab_0
pymc-base found but mismatch. Specification pkg: pymc-base==5.9.0=pyhd8ed1ab_0, Running pkg: pymc-base==5.9.1=pyhd8ed1ab_0

and compare to 5.8 where it is even faster, the only difference is pytensor

(pymc_env_5_9_1_test4) avicenna@avicenna:~/Desktop$ conda compare 5_8.yml
pymc found but mismatch. Specification pkg: pymc==5.8.0=hd8ed1ab_0, Running pkg: pymc==5.9.1=hd8ed1ab_0
pymc-base found but mismatch. Specification pkg: pymc-base==5.8.0=pyhd8ed1ab_0, Running pkg: pymc-base==5.9.1=pyhd8ed1ab_0
pytensor found but mismatch. Specification pkg: pytensor==2.15.0=py311hb755f60_0, Running pkg: pytensor==2.17.3=py311hb755f60_0
pytensor-base found but mismatch. Specification pkg: pytensor-base==2.15.0=py311h320fe9a_0, Running pkg: pytensor-base==2.17.3=py311h320fe9a_0

[ricardoV94]

The likely culprit is this PR: Allow batched parameters in MvNormal and MvStudentT distributions by ricardoV94 · Pull Request #6897 · pymc-devs/pymc · GitHub

To confirm, you found the biggest slowdown going from 5.9.0 to 5.9.1, right?

[iavicenna]

Correct, 5.9 is only about 2x times slower than 5.8 but 5.9.1 is infinitely slower than 5.9 (still after every other package is uniformized). And I have not encountered for univariate normals.

[ricardoV94]

What version of PyTensor is getting installed in 5.9.0 vs 5.9.1?

[iavicenna]

So when I do it by just specifying the pymc version to be 5.9.1, it installs the following:

pytensor                  2.13.1          py311ha02d727_0  
pytensor-base             2.17.3          py311h320fe9a_0    conda-forge

Then I also specified the pytensor and some other package versions (so that environments would be uniform between 5.9 and 5.9.1) in which both pytensor and base were 2.17.3. In both environments, the code was dead slow (both being the 2.9.1 with weird pytensor combination and the 2.9.1 with only 2.17.3 for pytensor and base). Could not install pymc 5.19.1 with the requirement pytensor-base==2.15.0=py311h320fe9a_0 (which is what comes with pymc 5.8), it was incompatible.

Here is the final 5.9.1 enviroment which identical with 5.9 except pymc

(pymc_env_5_9_1_test4) avicenna@avicenna:~/Desktop$ conda list
# packages in environment at /home/avicenna/miniconda3/envs/pymc_env_5_9_1_test4:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                  2_kmp_llvm    conda-forge
arviz                     0.16.1             pyhd8ed1ab_1    conda-forge
asttokens                 2.4.1              pyhd8ed1ab_0    conda-forge
atk-1.0                   2.38.0               hd4edc92_1    conda-forge
backcall                  0.2.0              pyh9f0ad1d_0    conda-forge
backports                 1.0                pyhd8ed1ab_3    conda-forge
backports.functools_lru_cache 1.6.5              pyhd8ed1ab_0    conda-forge
binutils_impl_linux-64    2.40                 hf600244_0    conda-forge
binutils_linux-64         2.40                 hbdbef99_2    conda-forge
blas                      2.119                  openblas    conda-forge
blas-devel                3.9.0           19_linux64_openblas    conda-forge
brotli                    1.1.0                hd590300_1    conda-forge
brotli-bin                1.1.0                hd590300_1    conda-forge
bzip2                     1.0.8                h7f98852_4    conda-forge
c-ares                    1.20.1               hd590300_1    conda-forge
ca-certificates           2023.7.22            hbcca054_0    conda-forge
cached-property           1.5.2                hd8ed1ab_1    conda-forge
cached_property           1.5.2              pyha770c72_1    conda-forge
cachetools                5.3.2              pyhd8ed1ab_0    conda-forge
cairo                     1.18.0               h3faef2a_0    conda-forge
certifi                   2023.7.22          pyhd8ed1ab_0    conda-forge
cloudpickle               3.0.0              pyhd8ed1ab_0    conda-forge
comm                      0.1.4              pyhd8ed1ab_0    conda-forge
cons                      0.4.6              pyhd8ed1ab_0    conda-forge
contourpy                 1.1.1           py311h9547e67_1    conda-forge
cycler                    0.12.1             pyhd8ed1ab_0    conda-forge
debugpy                   1.8.0           py311hb755f60_1    conda-forge
decorator                 5.1.1              pyhd8ed1ab_0    conda-forge
etuples                   0.3.9              pyhd8ed1ab_0    conda-forge
exceptiongroup            1.1.3              pyhd8ed1ab_0    conda-forge
executing                 2.0.1              pyhd8ed1ab_0    conda-forge
expat                     2.5.0                hcb278e6_1    conda-forge
fastprogress              1.0.3              pyhd8ed1ab_0    conda-forge
filelock                  3.13.1             pyhd8ed1ab_0    conda-forge
font-ttf-dejavu-sans-mono 2.37                 hab24e00_0    conda-forge
font-ttf-inconsolata      3.000                h77eed37_0    conda-forge
font-ttf-source-code-pro  2.038                h77eed37_0    conda-forge
font-ttf-ubuntu           0.83                 hab24e00_0    conda-forge
fontconfig                2.14.2               h14ed4e7_0    conda-forge
fonts-conda-ecosystem     1                             0    conda-forge
fonts-conda-forge         1                             0    conda-forge
fonttools                 4.43.1          py311h459d7ec_0    conda-forge
freetype                  2.12.1               h267a509_2    conda-forge
fribidi                   1.0.10               h36c2ea0_0    conda-forge
gcc                       12.3.0               h8d2909c_2    conda-forge
gcc_impl_linux-64         12.3.0               he2b93b0_2    conda-forge
gcc_linux-64              12.3.0               h76fc315_2    conda-forge
gdk-pixbuf                2.42.10              h829c605_4    conda-forge
gettext                   0.21.1               h27087fc_0    conda-forge
giflib                    5.2.1                h0b41bf4_3    conda-forge
graphite2                 1.3.13            h58526e2_1001    conda-forge
graphviz                  8.1.0                h28d9a01_0    conda-forge
gtk2                      2.24.33              h90689f9_2    conda-forge
gts                       0.7.6                h977cf35_4    conda-forge
gxx                       12.3.0               h8d2909c_2    conda-forge
gxx_impl_linux-64         12.3.0               he2b93b0_2    conda-forge
gxx_linux-64              12.3.0               h8a814eb_2    conda-forge
h5netcdf                  1.2.0              pyhd8ed1ab_0    conda-forge
h5py                      3.10.0          nompi_py311h3839ddf_100    conda-forge
harfbuzz                  8.2.1                h3d44ed6_0    conda-forge
hdf5                      1.14.2          nompi_h4f84152_100    conda-forge
icu                       73.2                 h59595ed_0    conda-forge
importlib-metadata        6.8.0              pyha770c72_0    conda-forge
importlib_metadata        6.8.0                hd8ed1ab_0    conda-forge
ipykernel                 6.26.0             pyhf8b6a83_0    conda-forge
ipython                   8.16.1             pyh0d859eb_0    conda-forge
jedi                      0.19.1             pyhd8ed1ab_0    conda-forge
joblib                    1.3.2              pyhd8ed1ab_0    conda-forge
jupyter_client            8.5.0              pyhd8ed1ab_0    conda-forge
jupyter_core              5.5.0           py311h38be061_0    conda-forge
kernel-headers_linux-64   2.6.32              he073ed8_16    conda-forge
keyutils                  1.6.1                h166bdaf_0    conda-forge
kiwisolver                1.4.5           py311h9547e67_1    conda-forge
krb5                      1.21.2               h659d440_0    conda-forge
lcms2                     2.15                 hb7c19ff_3    conda-forge
ld_impl_linux-64          2.40                 h41732ed_0    conda-forge
lerc                      4.0.0                h27087fc_0    conda-forge
libaec                    1.1.2                h59595ed_1    conda-forge
libblas                   3.9.0           19_linux64_openblas    conda-forge
libbrotlicommon           1.1.0                hd590300_1    conda-forge
libbrotlidec              1.1.0                hd590300_1    conda-forge
libbrotlienc              1.1.0                hd590300_1    conda-forge
libcblas                  3.9.0           19_linux64_openblas    conda-forge
libcurl                   8.4.0                hca28451_0    conda-forge
libdeflate                1.19                 hd590300_0    conda-forge
libedit                   3.1.20191231         he28a2e2_2    conda-forge
libev                     4.33                 h516909a_1    conda-forge
libexpat                  2.5.0                hcb278e6_1    conda-forge
libffi                    3.4.2                h7f98852_5    conda-forge
libgcc-devel_linux-64     12.3.0               h8bca6fd_2    conda-forge
libgcc-ng                 13.2.0               h807b86a_2    conda-forge
libgd                     2.3.3                h119a65a_9    conda-forge
libgfortran-ng            13.2.0               h69a702a_2    conda-forge
libgfortran5              13.2.0               ha4646dd_2    conda-forge
libglib                   2.78.0               hebfc3b9_0    conda-forge
libgomp                   13.2.0               h807b86a_2    conda-forge
libhwloc                  2.9.3           default_h554bfaf_1009    conda-forge
libiconv                  1.17                 h166bdaf_0    conda-forge
libjpeg-turbo             3.0.0                hd590300_1    conda-forge
liblapack                 3.9.0           19_linux64_openblas    conda-forge
liblapacke                3.9.0           19_linux64_openblas    conda-forge
libnghttp2                1.55.1               h47da74e_0    conda-forge
libnsl                    2.0.1                hd590300_0    conda-forge
libopenblas               0.3.24          pthreads_h413a1c8_0    conda-forge
libpng                    1.6.39               h753d276_0    conda-forge
librsvg                   2.56.3               h98fae49_0    conda-forge
libsanitizer              12.3.0               h0f45ef3_2    conda-forge
libsodium                 1.0.18               h36c2ea0_1    conda-forge
libsqlite                 3.43.2               h2797004_0    conda-forge
libssh2                   1.11.0               h0841786_0    conda-forge
libstdcxx-devel_linux-64  12.3.0               h8bca6fd_2    conda-forge
libstdcxx-ng              13.2.0               h7e041cc_2    conda-forge
libtiff                   4.6.0                ha9c0a0a_2    conda-forge
libtool                   2.4.7                h27087fc_0    conda-forge
libuuid                   2.38.1               h0b41bf4_0    conda-forge
libwebp                   1.3.2                h658648e_1    conda-forge
libwebp-base              1.3.2                hd590300_0    conda-forge
libxcb                    1.15                 h0b41bf4_0    conda-forge
libxml2                   2.11.5               h232c23b_1    conda-forge
libzlib                   1.2.13               hd590300_5    conda-forge
llvm-openmp               17.0.3               h4dfa4b3_0    conda-forge
logical-unification       0.4.6              pyhd8ed1ab_0    conda-forge
matplotlib-base           3.8.0           py311h54ef318_2    conda-forge
matplotlib-inline         0.1.6              pyhd8ed1ab_0    conda-forge
minikanren                1.0.3              pyhd8ed1ab_0    conda-forge
mkl                       2022.2.1         h84fe81f_16997    conda-forge
mkl-service               2.4.0           py311hb711fc7_0    conda-forge
multipledispatch          0.6.0                      py_0    conda-forge
munkres                   1.1.4              pyh9f0ad1d_0    conda-forge
ncurses                   6.4                  h59595ed_2    conda-forge
nest-asyncio              1.5.8              pyhd8ed1ab_0    conda-forge
numpy                     1.25.2          py311h64a7726_0    conda-forge
openblas                  0.3.24          pthreads_h7a3da1a_0    conda-forge
openjpeg                  2.5.0                h488ebb8_3    conda-forge
openssl                   3.1.4                hd590300_0    conda-forge
packaging                 23.2               pyhd8ed1ab_0    conda-forge
pandas                    2.1.2           py311h320fe9a_0    conda-forge
pango                     1.50.14              ha41ecd1_2    conda-forge
parso                     0.8.3              pyhd8ed1ab_0    conda-forge
pcre2                     10.40                hc3806b6_0    conda-forge
pexpect                   4.8.0              pyh1a96a4e_2    conda-forge
pickleshare               0.7.5                   py_1003    conda-forge
pillow                    10.1.0          py311ha6c5da5_0    conda-forge
pip                       23.3.1             pyhd8ed1ab_0    conda-forge
pixman                    0.42.2               h59595ed_0    conda-forge
platformdirs              3.11.0             pyhd8ed1ab_0    conda-forge
prompt-toolkit            3.0.39             pyha770c72_0    conda-forge
prompt_toolkit            3.0.39               hd8ed1ab_0    conda-forge
psutil                    5.9.5           py311h459d7ec_1    conda-forge
pthread-stubs             0.4               h36c2ea0_1001    conda-forge
ptyprocess                0.7.0              pyhd3deb0d_0    conda-forge
pure_eval                 0.2.2              pyhd8ed1ab_0    conda-forge
pygments                  2.16.1             pyhd8ed1ab_0    conda-forge
pymc                      5.9.1                hd8ed1ab_0    conda-forge
pymc-base                 5.9.1              pyhd8ed1ab_0    conda-forge
pyparsing                 3.1.1              pyhd8ed1ab_0    conda-forge
pytensor                  2.17.3          py311hb755f60_0    conda-forge
pytensor-base             2.17.3          py311h320fe9a_0    conda-forge
python                    3.11.6          hab00c5b_0_cpython    conda-forge
python-dateutil           2.8.2              pyhd8ed1ab_0    conda-forge
python-graphviz           0.20.1             pyh22cad53_0    conda-forge
python-tzdata             2023.3             pyhd8ed1ab_0    conda-forge
python_abi                3.11                    4_cp311    conda-forge
pytz                      2023.3.post1       pyhd8ed1ab_0    conda-forge
pyzmq                     25.1.1          py311h34ded2d_2    conda-forge
readline                  8.2                  h8228510_1    conda-forge
scikit-learn              1.3.2           py311hc009520_1    conda-forge
scipy                     1.11.3          py311h64a7726_1    conda-forge
setuptools                68.2.2             pyhd8ed1ab_0    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
spyder-kernels            2.4.4           unix_pyh707e725_0    conda-forge
stack_data                0.6.2              pyhd8ed1ab_0    conda-forge
sysroot_linux-64          2.12                he073ed8_16    conda-forge
tbb                       2021.10.0            h00ab1b0_2    conda-forge
threadpoolctl             3.2.0              pyha21a80b_0    conda-forge
tk                        8.6.13               h2797004_0    conda-forge
toolz                     0.12.0             pyhd8ed1ab_0    conda-forge
tornado                   6.3.3           py311h459d7ec_1    conda-forge
traitlets                 5.13.0             pyhd8ed1ab_0    conda-forge
typing-extensions         4.8.0                hd8ed1ab_0    conda-forge
typing_extensions         4.8.0              pyha770c72_0    conda-forge
tzdata                    2023c                h71feb2d_0    conda-forge
wcwidth                   0.2.9              pyhd8ed1ab_0    conda-forge
wheel                     0.41.3             pyhd8ed1ab_0    conda-forge
wurlitzer                 3.0.3              pyhd8ed1ab_0    conda-forge
xarray                    2023.10.1          pyhd8ed1ab_0    conda-forge
xarray-einstats           0.6.0              pyhd8ed1ab_0    conda-forge
xorg-kbproto              1.0.7             h7f98852_1002    conda-forge
xorg-libice               1.1.1                hd590300_0    conda-forge
xorg-libsm                1.2.4                h7391055_0    conda-forge
xorg-libx11               1.8.7                h8ee46fc_0    conda-forge
xorg-libxau               1.0.11               hd590300_0    conda-forge
xorg-libxdmcp             1.1.3                h7f98852_0    conda-forge
xorg-libxext              1.3.4                h0b41bf4_2    conda-forge
xorg-libxrender           0.9.11               hd590300_0    conda-forge
xorg-renderproto          0.11.1            h7f98852_1002    conda-forge
xorg-xextproto            7.3.0             h0b41bf4_1003    conda-forge
xorg-xproto               7.0.31            h7f98852_1007    conda-forge
xz                        5.2.6                h166bdaf_0    conda-forge
zeromq                    4.3.5                h59595ed_0    conda-forge
zipp                      3.17.0             pyhd8ed1ab_0    conda-forge
zlib                      1.2.13               hd590300_5    conda-forge
zstd                      1.5.5                hfc55251_0    conda-forge

[ricardoV94]

The next step would be to look at the functions that actually get compiled.

import pytensor
import pymc as pm

with pm.Model() as m:
   ...

print("Logp-dlogp fn")
pytensor.dprint(
    m.logp_dlogp_function()._pytensor_function
)

print("Trace fn")
pytensor.dprint(
    m.compile_fn(inputs=m.value_vars, outs=m.unobserved_value_vars, point_fn=False)
)

[ricardoV94]

Wait did you say that when pytensor=2.17.3 both pymc=5.9.0 and pymc=5.9.1 are dead slow?

That combination of pytensor=2.13.1 and pytensor-base=2.17.3 is very weird, and shouldn’t work. How did you arrive at a situation where you get that combo?