Pymc3 stack Docker

Questions
1

Hey all,

I am trying to get the following example to run:
https://aesara.readthedocs.io/en/latest/tutorial/adding.html#adding-two-matrices

This works:
import pymc3 as pm
if pm.version == ‘4.0’:
import aesara as th # efficent tensors
import aesara.tensor as tt # tensor control
else:
import theano as th # efficent tensors
import theano.tensor as tt # tensor control

then this works:

declare two symbolic floating-point scalars

a = tt.dscalar()
b = tt.dscalar()

create a simple expression

c = a + b

then this bombs:
f = th.function([a, b], c)

like this:
/opt/conda/lib/python3.9/site-packages/theano/compile/function/pfunc.py in _pfunc_param_to_in(param, strict, allow_downcast)
541 elif isinstance(param, In):
542 return param
→ 543 raise TypeError(f"Unknown parameter type: {type(param)}")

I wanted it to work on my mac, but clang is giving me fits, so I thought I would try a container.

I found this:

but it has not been updated in a long time and the base container’s python is now v3.9. I think Pymc is just tested on v3.8 correct?

I have figured out that most releases of Pymc have a dependencies complete with versions… I just can’t figure out if I want a functioning full stack what I need to do. What is the recommended full stack and base python that I need on an ubuntu image? can we use miniconda or something like it rather than anaconda?

2

sorry, forgot to post my dockerfile:

FROM jupyter/minimal-notebook

if you try to install right on top of this, it will complain that you don’t have a c compiler

USER root

ENV DEBIAN_FRONTEND noninteractive

RUN apt-get update

RUN apt-get -y install gcc mono-mcs

RUN rm -rf /var/lib/apt/lists/*

USER jovyan

#install all the prereqs…

RUN conda install mkl-service

RUN pip install cachetools==4.2.1

RUN pip install dill

RUN pip install fastprogress==0.2.0

RUN pip install numpy

RUN pip install pandas

RUN pip install patsy==0.5.1

RUN pip install scipy

RUN pip install semver==2.13.0

RUN pip install theano-pymc==1.1.2

RUN pip install typing-extensions==3.7.4

do the thing…

RUN conda install -c conda-forge pymc3

#this version gets upgraded to something incompatable when you do the command above, so need to downgrade it

RUN pip install arviz===0.11.0

matplotlib nonsense

ENV XDG_CACHE_HOME /home/$NB_USER/.cache/

ENV MPLBACKEND=Agg

for prettier default plot styling

RUN pip install seaborn

Import matplotlib the first time to build the font cache.

RUN python -c “import matplotlib.pyplot”

ENV PYTHONPATH $PYTHONPATH:"$HOME"

3

ok, so I did this:
pip install git+https://github.com/aesara-devs/aesara

that got me further, it looks like everything on this page now works:

I do have the following warning:
WARNING (aesara.configdefaults): g++ not detected ! Aesara will be unable to execute optimized C-implementations (for both CPU and GPU) and will default to Python implementations. Performance will be severely degraded. To remove this warning, set Aesara flags cxx to an empty string.
WARNING (aesara.tensor.blas): Using NumPy C-API based implementation for BLAS functions.

any idea how to fix this?

1 Like
4

I am also struggling with creating a docker container that has pymc3 installed correctly (such that the warning Using NumPy C-API based implementation for BLAS functions does not appear).

If anyone has succeeded, please post your docker file!

5

The docs strongly recommend using conda, so I tried this:

FROM continuumio/anaconda-pkg-build
RUN conda install -c conda-forge pymc3 mkl-service
RUN conda update pymc3

but then pymc3 raises these 2 warnings:

WARNING (theano.configdefaults): g++ not detected ! Theano will be unable to execute optimized C-
WARNING (theano.tensor.blas): Using NumPy C-API based implementation for BLAS functions.

Starting with a basic python image and using pip:

FROM python:3.9
RUN apt-get update
RUN apt-get -y install build-essential
RUN pip install pymc3==3.11.4

results in just the 2nd warning:

WARNING (theano.tensor.blas): Using NumPy C-API based implementation for BLAS functions.

…and seems to work just fine. @twiecki says this warning is ok to ignore:

That said, if anyone has a recipe for a better setup, please let me know.

6

I noticed that switching to “blas” from “openblas” on my linux system made a difference at some point; maybe worth a try? It might at least help some of the warnings.

7

I think i cleared the BLAS issues. See the conda build/compose files here. GitHub - alephinsights/docker-pymc-gpu

I’m now stuck with complaints about
pygpu.gpuarray.GpuArrayException: b'Could not load "libnvrtc.so"

any ideas on this one?