How to fix ImportError or Intel MKL Error during PyMC3 import in pyinstaller executable

Questions
1

Hello,

I am attempting to create a standalone executable tool based on a PyMC3 sampler with Pyinstaller. As a .py script, the program runs fine, but I have not yet been able to run it once it is compiled to an executable.

I have come across generally two types of error messages during my trouble shooting process:

If compiled in some virtual environment with Python 3.7.x or 3.8.x, running the executable yields
INTEL MKL ERROR: The specified module could not be found. mkl_intel_thread.1.dll. Intel MKL FATAL ERROR: Cannot load mkl_intel_thread.1.dll.
with no traceback.

If compiled in a virtual environment with Python 3.9.x, running the executable yields
ImportError: cannot import name 'ccompiler' from partially initialized module 'numpy.distutils' (most likely due to a circular import) (C:\...\AppData\Local\Temp\_MEI238682\numpy\distutils\__init__.pyc)
with traceback
File "*[pymc script]*.py", line 1, in <module> File "PyInstaller\loader\pyimod03_importers.py", line 540, in exec_module File "pymc3\__init__.py", line 23, in <module> File "PyInstaller\loader\pyimod03_importers.py", line 540, in exec_module File "theano\__init__.py", line 83, in <module> File "PyInstaller\loader\pyimod03_importers.py", line 540, in exec_module File "theano\scalar\__init__.py", line 1, in <module> File "PyInstaller\loader\pyimod03_importers.py", line 540, in exec_module File "theano\scalar\basic.py", line 22, in <module> File "PyInstaller\loader\pyimod03_importers.py", line 540, in exec_module File "theano\printing.py", line 18, in <module> File "PyInstaller\loader\pyimod03_importers.py", line 540, in exec_module File "theano\compile\__init__.py", line 21, in <module> File "PyInstaller\loader\pyimod03_importers.py", line 540, in exec_module File "theano\compile\mode.py", line 21, in <module> File "PyInstaller\loader\pyimod03_importers.py", line 540, in exec_module File "theano\link\c\basic.py", line 19, in <module> File "PyInstaller\loader\pyimod03_importers.py", line 540, in exec_module File "theano\link\c\cmodule.py", line 23, in <module> File "PyInstaller\loader\pyimod03_importers.py", line 540, in exec_module File "numpy\distutils\__init__.py", line 24, in <module>

showing that it failed during pymc3/theano import. This happens at the pymc3 import regardless of the order of the import steps in the script.

I also added an import step for that ccompiler from numpy to see what would happen leading to
ImportError: cannot import name 'log' from partially initialized module 'numpy.distutils' (most likely due to a circular import) (C:\...\AppData\Local\Temp\_MEI170202\numpy\distutils\__init__.pyc)
and then with an import step for that…
ModuleNotFoundError: No module named 'numpy.distutils.misc_util'

Other 3.8.x venvs yield
ModuleNotFoundError: No module named 'theano.tensor.shared_randomstreams'

I do not really know what would specifically cause any of these errors.

These are run in virtual environments containing only the necessary libraries. Updating libraries did not help. I have successfully used pyinstaller for non-MCMC programs before. I tried both onefile and one-directory options for the executable format.

Has anyone run into similar problems when running pymc3 through pyinstaller? Is there a better route I should be taking for a simple downloadable mcmc tool? My problems could possibly be more related to numpy or pyinstaller configuration, but since the error occurred during PyMC import I figured it would be worth asking here.

Thanks :slight_smile:

2

Any reason you can’t use conda?

3

I have been using conda for environment management and library installation. Unless you mean something else?

4

In your original question you wrote something about PyInstaller and virtualenv. What do you need those for if you’re using conda?

5

I just meant the conda environments, apologies for the confusion. Does conda have a method to compile a python script to an executable with no dependencies? Do you mean a conda constructor? I’m specifically creating something to distribute to a few coworkers without the need to set up python

6

I don’t think that will be possible as Theano/Aesara needs a compiler.

7

I ran into this problem too. I had the same error as yours when trying to create an exe tool. Have you figured it out yet?