Hi, I tried to install pymc4 using
%pip install pymc --pre
And got the following error. I’m wondering if anyone has had any experience with this.
Collecting pymc
Downloading pymc-2.3.8.tar.gz (385 kB)
|████████████████████████████████| 385 kB 31.8 MB/s
Preparing metadata (setup.py) … done
Building wheels for collected packages: pymc
Building wheel for pymc (setup.py) … error
ERROR: Command errored out with exit status 1:
command: /home/ec2-user/anaconda3/envs/pytorch_latest_p36/bin/python -u -c ‘import io, os, sys, setuptools, tokenize; sys.argv[0] = ‘"’"’/tmp/pip-install-j7jugav9/pymc_4b69c960644e4f5482bae7edc3dfcecd/setup.py’"’"’; file=’"’"’/tmp/pip-install-j7jugav9/pymc_4b69c960644e4f5482bae7edc3dfcecd/setup.py’"’"’;f = getattr(tokenize, ‘"’"‘open’"’"’, open)(file) if os.path.exists(file) else io.StringIO(’"’"‘from setuptools import setup; setup()’"’"’);code = f.read().replace(’"’"’\r\n’"’"’, ‘"’"’\n’"’"’);f.close();exec(compile(code, file, ‘"’"‘exec’"’"’))’ bdist_wheel -d /tmp/pip-wheel-e2eqdin_
cwd: /tmp/pip-install-j7jugav9/pymc_4b69c960644e4f5482bae7edc3dfcecd/
1 Like
Youre accidentally installing PyMC 2.3.8
Try using using this command pip install pymc==4.0.0b6
1 Like
Hi, @RavinKumar
Thank you, this works for me on Sagemaker too. But once installing, i see the warning below about potential downgrade of performance.
WARNING (theano.configdefaults): g++ not detected ! Theano will be unable to execute optimized C-implementations (for both CPU and GPU) and will default to Python implementations. Performance will be severely degraded. To remove this warning, set Theano flags cxx to an empty string.
WARNING (theano.tensor.blas): Using NumPy C-API based implementation for BLAS functions.
WARNING (aesara.configdefaults): g++ not detected ! Aesara will be unable to execute optimized C-implementations (for both CPU and GPU) and will default to Python implementations. Performance will be severely degraded. To remove this warning, set Aesara flags cxx to an empty string.
WARNING (aesara.tensor.blas): Using NumPy C-API based implementation for BLAS functions.
Any hints on how to solve them?
Thanks again!
Thank you for your response, however I still can’t install pymc4. Here’s the error I get:
ERROR: Could not find a version that satisfies the requirement pymc==4.0.0b6 (from versions: 2.3.5.py27-macosx-x86_64, 2.3.5.py34-macosx-x86_64, 2.3.6.py27-macosx-x86_64, 2.3.6.py34-macosx-x86_64, 2.3.6.py35-macosx-x86_64, 2.0, 2.1b0, 2.2, 2.3, 2.3.1, 2.3.2, 2.3.3, 2.3.4, 2.3.5, 2.3.6, 2.3.8)
ERROR: No matching distribution found for pymc==4.0.0b6
Looks like when I try to install “pymc”, it only searches through pymc2 versions.
Hi, it didn’t work for me at all. What was the command you ran which gave you the above warnings?
Hi, @payamphysics I am using the command that @RavinKumar shared: pip install pymc==4.0.0b6
From your error message, it seems that your sagemaker is running on Python2 or lower Python3, i guess this might be the reason that you see no such package found, as i guess pymc only supports higher version of Python3, e…g Python3.7+?
The engine I’m using on sagemaker is “conda_pytorch_latest_p36”, which I always supposed to be Python 3.6. What’s the one you use?
Our default version on sagemaker is Python3.9, but this is just my guess that this might be related to python version.
I don’t see documentation on pymc mentioning python version.
Here’s the full list of the options I have available:
Would you be able to let me know if the one you use is in this list, and if it is, which one it is?
Hey folks, the issues you’re having with sagemaker due to the complexity that sagemaker introducecs when they ironically try and make things “simple”
Basically sagemaker is an EC2 instance that is magically created when you start sagemaker. Now because its EC2 it has its own base OS with its own python and all that, but also is missing additional libraries like g++ and other compilers because of the the next point.
To make sagemaker “easy for data scientists” amazon ships them with preconfigured kernels, the one that @payamphysics posted. They optimize for “popular DS” as you can see with the tensorflow and pytorch environments.
Now when you try and install PyMC, besides ensuring youre installling v4, you also have to ensure that its being installed in the conda environment that is right for you, and also manually install any other dependencies such as lower level compilers. Unfortunately this is really annoying on sagemaker because you can’t directly use a terminal.
Either you need to create a custom kernel using a series of commands in one of other sagemakers UIs, or every time you create a new instance you’ll need to run setup scripts. This is what I had to do when I used to use sagemaker, and admittedly it was not fun. This is one of those things where by making some things easier, amazon made other things REALLY hard. I stopped using Sagemaker shortly after due to this reason, and switched to EC2 instances where I ssh in and setup a notebook environment. This ended up being simpler in the long run for me
What you can try as a test is installing PyMC in an EC2 instance just to double check you can get everything working. If you get it working there, then its a matter of just configuring sagemaker correctly
The sagemaker docs explain the custom kernel stuff more. I hope this explanation helped
1 Like
Hi Ravin, Thank you for your detailed explanations. I will try to follow them to see what I can get. As a side point, it is possible to have a terminal on sagmaker. In the list I posted above (the image), at the bottom of the list there’s a “Terminal” option (which is not shown in the image I posted above).
I am using a customised image made by our engineers in the team, so not any of them actually. But it is python3.9
Thanks for your detailed reply @RavinKumar Will try to use conda for installation then!
PyMC 4.0.0b6 must be installed from pip, as it was never released on conda. There is one older version there (b5) but that is not recommended as it had a quite serious bug
I created a conda-forge package for 4.0.0b6 now.
6 Likes
@payamphysics I would try the conda_python3 option.
In the notebook you’re using run import sys; sys.version. That will tell us what version of python it is running. If it is running python 3.6, then you are out of luck. I’ve just checked via docker, and pymc 4 does not install on python 3.6. (It tries installing 2.3.8)
# contents of file "Dockerfile"
FROM python:3.6-slim-buster # try 3.7 or upwards to see that it works there
RUN pip install pymc
This is a convenient way to check whether python applications will install on a given python version and cpu architecture.
Run docker build -t py36 . from that directory. If you’re on an M1 mac, then you’d add --platform linux/amd64 at the end to ensure that you’re targeting the same architecture that Amazon is running (x86_64).
Thank you very much Thomas for this very informative post! I work on AWS Sagemaker and in fact, my Python version is 3.6.13. The reason I have to use Sagemaker is that I don’t have GPU on my personal (work) computer.
There’s a current Docker image in the works that might also be helpful: Update Docker image to `v4` · Issue #5323 · pymc-devs/pymc · GitHub
1 Like
Thank you! I will try that out.
