Hello together,
I’ve built a model (successfully) using pymc and for this model, I want to use the nutpie
sampler (mainly due to performance reasons, as my model runs in 10 minutes using nutpie
compared to multiple hours using the default pymc
sampler).
My challenge:
I am currently in the process to bring my model into production - meaning I want to schedule it via Airflow which in our setup requires using a Docker container.
I managed to install nutpie
in Docker (having quite some headache) using the following Dockerfile:
FROM python:3.11.2-slim-buster
# - - - - - - UNIX-LEVEL INSTALLTIONS - - - - - - - -
USER root
RUN apt update && apt upgrade -y \
&& apt -y install --no-install-recommends -qq \
curl ca-certificates make g++ git vim-tiny openssh-client \
unzip tzdata htop \
&& apt clean && rm -rf /var/lib/apt/lists/*
RUN apt-get update && \
apt-get -y install gcc mono-mcs && \
rm -rf /var/lib/apt/lists/*
RUN apt-get update && apt-get install -y \
build-essential \
libedit-dev \
llvm
# - - - - - - - - - - - - - - - - - - - - - - - - - -
# - - - - - - INSTALL PACKAGES - - - - - - - - - - -
RUN pip install --upgrade pip
RUN pip install "pymc>=5"
RUN pip install "google-cloud-storage" \
"google-cloud-bigquery" \
"arviz" \
"matplotlib" \
"numpy" \
"pandas" \
"seaborn" \
"rich" \
"pandas_gbq"
# - - - - - - - - - - - - - - - - - - - - - - - - - -
# - - - - - - IMPORT FILES AND GRANT ACCESS - - - - -
COPY . /app
WORKDIR /app
RUN useradd -ms /bin/bash admin
RUN chown -R admin:admin /app
RUN chmod 755 /app
USER admin
# - - - - - - - - - - - - - - - - - - - - - - - - - -
# - - - - - - INSTALL NUPIE SAMPLER - - - - - - - - -
RUN curl https://sh.rustup.rs -sSf | bash -s -- -y
# RUN apt -y install gcc
ENV PATH="$PATH:/home/admin/.cargo/bin"
#llvm, gfortran
RUN pip install 'nutpie[pymc]'
# - - - - - - - - - - - - - - - - - - - - - - - - - -
CMD ["python", "run_model.py"]
Still when I run a model, I get an error that refers to missing dependencies - but in the web I just can’t find which dependencies are missing:
File "/app/test_nutpie_in_docker.py", line 11, in <module>
compiled_model = nutpie.compile_pymc_model(model)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/admin/.local/lib/python3.11/site-packages/nutpie/__init__.py", line 7, in compile_pymc_model
raise ValueError("Missing dependencies for pymc. `import nutpie.compile_pymc` to see error.")
ValueError: Missing dependencies for pymc. `import nutpie.compile_pymc` to see error.
Can someone please help here?
Thanks a lot in advance