PyTensor Problem with PyMC-BART in Google Colab?

David_Arredondo · September 6, 2024, 9:51pm

Short Description

Hi all,

I’m trying to run the first BART model example from Bayesian Analysis with Python in Google Colab.

I’m getting some errors that I think are related to pytensor? I’m an absolute novice, so I have no idea. Also, apologies for all the text below; I just don’t know what’s relevant and what’s not.

When I run the following:

with pm.Model() as model_pen:
    σ = pm.HalfNormal("σ", 1)
    μ = pmb.BART("μ", X, Y, m=50)
    y = pm.Normal("y", mu=μ, sigma=σ, observed=Y)
    idata_pen = pm.sample()

I get a long series of errors. One in particular sorta repeats for while. See below.

----
/usr/local/lib/python3.10/site-packages/pytensor/tensor/random/op.py:84: FutureWarning: ndim_supp is deprecated. Provide signature instead.
  warnings.warn(
/usr/local/lib/python3.10/site-packages/pytensor/tensor/random/op.py:94: FutureWarning: ndims_params is deprecated. Provide signature instead.
  warnings.warn(
ERROR (pytensor.graph.rewriting.basic): Rewrite failure due to: constant_folding
ERROR (pytensor.graph.rewriting.basic): node: Cast{float32}(1)
ERROR (pytensor.graph.rewriting.basic): TRACEBACK:
ERROR (pytensor.graph.rewriting.basic): Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/pytensor/graph/rewriting/basic.py", line 1909, in process_node
    replacements = node_rewriter.transform(fgraph, node)
  File "/usr/local/lib/python3.10/site-packages/pytensor/graph/rewriting/basic.py", line 1081, in transform
    return self.fn(fgraph, node)
  File "/usr/local/lib/python3.10/site-packages/pytensor/tensor/rewriting/basic.py", line 1121, in constant_folding
    thunk = node.op.make_thunk(node, storage_map, compute_map, no_recycling=[])
  File "/usr/local/lib/python3.10/site-packages/pytensor/link/c/op.py", line 119, in make_thunk
    return self.make_c_thunk(node, storage_map, compute_map, no_recycling)
  File "/usr/local/lib/python3.10/site-packages/pytensor/link/c/op.py", line 84, in make_c_thunk
    outputs = cl.make_thunk(
  File "/usr/local/lib/python3.10/site-packages/pytensor/link/c/basic.py", line 1182, in make_thunk
    cthunk, module, in_storage, out_storage, error_storage = self.__compile__(
  File "/usr/local/lib/python3.10/site-packages/pytensor/link/c/basic.py", line 1103, in __compile__
    thunk, module = self.cthunk_factory(
  File "/usr/local/lib/python3.10/site-packages/pytensor/link/c/basic.py", line 1627, in cthunk_factory
    module = cache.module_from_key(key=key, lnk=self)
  File "/usr/local/lib/python3.10/site-packages/pytensor/link/c/cmodule.py", line 1255, in module_from_key
    module = lnk.compile_cmodule(location)
  File "/usr/local/lib/python3.10/site-packages/pytensor/link/c/basic.py", line 1528, in compile_cmodule
    module = c_compiler.compile_str(
  File "/usr/local/lib/python3.10/site-packages/pytensor/link/c/cmodule.py", line 2654, in compile_str
    raise CompileError(
pytensor.link.c.exceptions.CompileError: Compilation failed (return status=1):
/usr/local/bin/g++ -shared -g -O3 -fno-math-errno -Wno-unused-label -Wno-unused-variable -Wno-write-strings -Wno-c++11-narrowing -fno-exceptions -fno-unwind-tables -fno-asynchronous-unwind-tables -DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION -m64 -fPIC -I/usr/local/lib/python3.10/dist-packages/numpy/core/include -I/usr/include/python3.10 -I/usr/local/lib/python3.10/site-packages/pytensor/link/c/c_code -L/usr/lib/x86_64-linux-gnu -L/usr/lib -fvisibility=hidden -o /root/.pytensor/compiledir_Linux-6.1.85+-x86_64-with-glibc2.35-x86_64-3.10.12-64/tmpa484ve2i/mbb1a558a69b3834cbd8bb5160f53dcb1edcf00af31190d19c71508ba0bcc380a.so /root/.pytensor/compiledir_Linux-6.1.85+-x86_64-with-glibc2.35-x86_64-3.10.12-64/tmpa484ve2i/mod.cpp -lpython3.10
In file included from /usr/include/python3.10/Python.h:8,
                 from /root/.pytensor/compiledir_Linux-6.1.85+-x86_64-with-glibc2.35-x86_64-3.10.12-64/tmpa484ve2i/mod.cpp:1:
/usr/include/python3.10/pyconfig.h:3:12: fatal error: x86_64-linux-gnu/python3.10/pyconfig.h: No such file or directory
    3 | #  include <x86_64-linux-gnu/python3.10/pyconfig.h>
      |            ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.

----

#### There's lots of other stuff, but here' the final error message:

----
ImportError: Version check of the existing lazylinker compiled file. Looking for version 0.212, but found None. Extra debug information: force_compile=False, _need_reload=True

During handling of the above exception, another exception occurred:

CompileError                              Traceback (most recent call last)
[/usr/local/lib/python3.10/site-packages/pytensor/link/c/cmodule.py](https://localhost:8080/#) in compile_str(module_name, src_code, location, include_dirs, lib_dirs, libs, preargs, py_module, hide_symbols)
   2652             # difficult to read.
   2653             # compile_stderr = compile_stderr.replace("\n", ". ")
-> 2654             raise CompileError(
   2655                 f"Compilation failed (return status={status}):\n{' '.join(cmd)}\n{compile_stderr}"
   2656             )

CompileError: Compilation failed (return status=1):
/usr/local/bin/g++ -shared -g -Wno-c++11-narrowing -fno-exceptions -fno-unwind-tables -fno-asynchronous-unwind-tables -DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION -m64 -fPIC -I/usr/local/lib/python3.10/dist-packages/numpy/core/include -I/usr/include/python3.10 -I/usr/local/lib/python3.10/site-packages/pytensor/link/c/c_code -L/usr/lib/x86_64-linux-gnu -L/usr/lib -fvisibility=hidden -o /root/.pytensor/compiledir_Linux-6.1.85+-x86_64-with-glibc2.35-x86_64-3.10.12-64/lazylinker_ext/lazylinker_ext.so /root/.pytensor/compiledir_Linux-6.1.85+-x86_64-with-glibc2.35-x86_64-3.10.12-64/lazylinker_ext/mod.cpp -lpython3.10
In file included from /usr/include/python3.10/Python.h:8,
                 from /usr/local/lib/python3.10/site-packages/pytensor/link/c/c_code/pytensor_mod_helper.h:4,
                 from /root/.pytensor/compiledir_Linux-6.1.85+-x86_64-with-glibc2.35-x86_64-3.10.12-64/lazylinker_ext/mod.cpp:1:
/usr/include/python3.10/pyconfig.h:3:12: fatal error: x86_64-linux-gnu/python3.10/pyconfig.h: No such file or directory
    3 | #  include <x86_64-linux-gnu/python3.10/pyconfig.h>
      |            ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.

----

Sorry for including so much text, but these errors are all way above my understanding, and I don’t know what’s relevant. Maybe the problem is colab? As in, I might not have the appropriate version of some library?

Anyway, all of my code is below.

My code

#downloads conda
!pip install -q condacolab
import condacolab
condacolab.install()

#makes sure I have pymc v5
!conda install -c conda-forge "pymc>=5"

#install pymc_bart
!conda install -c conda-forge pymc-bart

import arviz as az
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import pymc as pm
import pymc_bart as pmb

#data
penguins = pd.read_csv("https://raw.githubusercontent.com/thedarredondo/data-science-fundamentals/main/Data/penguins.csv").dropna()
Y = penguins["body_mass"]
pen = penguins
pen['sex'] = pen['sex'].replace(['male', 'female'],[0, 1])
pen['species'] = pen['species'].replace(['Adelie', 'Chinstrap', 'Gentoo'],[0, 1, 2])
X = pen[["flipper_length", "bill_depth", "bill_length"]]

#the textbook example
with pm.Model() as model_pen:
    σ = pm.HalfNormal("σ", 1)
    μ = pmb.BART("μ", X, Y, m=50)
    y = pm.Normal("y", mu=μ, sigma=σ, observed=Y)
    idata_pen = pm.sample()

ricardoV94 · September 7, 2024, 2:45am

PyMC is already installed by default in Colab. You can use pip install for pymc-bart if it’s not already there. I have never tried conda on Colab (nor needed it)

David_Arredondo · September 8, 2024, 5:24pm

You’re right pymc is already installed–I think I was trying to see if manually installing it would get rid of the following problem:

import pymc as pm
import pymc-bart as pmb

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-2-0d8bfe5a7b80> in <cell line: 6>()
      4 import pandas as pd
      5 import pymc as pm
----> 6 import pymc_bart as pmb

1 frames
/usr/local/lib/python3.10/dist-packages/pymc_bart/bart.py in <module>
     23 import pytensor.tensor as pt
     24 from pandas import DataFrame, Series
---> 25 from pymc.distributions.distribution import Distribution, _support_point
     26 from pymc.logprob.abstract import _logprob
     27 from pytensor.tensor.random.op import RandomVariable

ImportError: cannot import name '_support_point' from 'pymc.distributions.distribution' (/usr/local/lib/python3.10/dist-packages/pymc/distributions/distribution.py)

---------------------------------------------------------------------------
NOTE: If your import is failing due to a missing package, you can
manually install dependencies using either !pip or !apt.

To view examples of installing some common dependencies, click the
"Open Examples" button below.
---------------------------------------------------------------------------

. . . except now that problem is happening no matter what I do, install wise.

Any suggestions?

cluhmann · September 24, 2024, 7:38pm

Pip installing pymc-bart seems to work ok for me (at least it allows me to then import pymc_bart).

David_Arredondo · September 26, 2024, 6:15pm

Thanks!

And yep, works for me now! It was either me doing something crazy with conda, or the update to pymc 5.16 in colab.

Topic		Replies	Views
[bug] error in the warning of pmb.plot_dependence Development bug	7	239	November 2, 2023
Problem to load trained BART model for prediction v5 bart	5	681	December 6, 2023
Save and Load a BART model v5 bart	8	488	August 22, 2024
Is it a shape error by BART or my lack of knowledge? v5 modeling	6	1175	January 26, 2023
Bayesian VAR with missing data: PyTensor error when sampling with blackjax or numpyro v5 bug , modeling	3	399	June 2, 2023

PyTensor Problem with PyMC-BART in Google Colab?

Short Description

My code

Related Topics