# Introducing Gumbi: the Gaussian Process Model Building Interface
**URL:** https://discourse.pymc.io/t/introducing-gumbi-the-gaussian-process-model-building-interface/8377
**Category:** Sharing
**Tags:** gaussian\_process
**Created:** [November 28, 2021, 11:10pm UTC](https://discourse.pymc.io/t/introducing-gumbi-the-gaussian-process-model-building-interface/8377 "2021-11-28T23:10:25Z")
**Posts on this page:** 14
**Page:** 1
### Author:  [@BioGoertz](https://discourse.pymc.io/u/BioGoertz)
#### Post date: [November 28, 2021, 11:10pm UTC](https://discourse.pymc.io/t/introducing-gumbi-the-gaussian-process-model-building-interface/8377/1 "2021-11-28T23:10:25Z")
Hello Pymc Community!
I wanted to share a project I’ve been working on: [Gumbi, the Gaussian Process Model Building Interface](https://github.com/JohnGoertz/Gumbi). Inspired by what [Bambi](https://bambinos.github.io/bambi/main/index.html) does for generalized linear models, the goal of Gumbi is to make it easy and intuitive to quickly prototype Gaussian Process models. Specifically designed for tabular data (pandas DataFrames), Gumbi takes care of transforming, standardizing, and reshaping data before feeding it into an auto-built Pymc model, and then un-doing all of that wrangling when it comes to making predictions. Gumbi also includes visualization tools for easy plotting of predictions. For now there is only a Pymc backend, but I may add a GPFlow backend as well.
As shown in the quickstart, Gumbi reduces building, fitting, predicting, and visualizing a model down to a few simple commands.
> Read in some data and store it as a Gumbi `DataSet`:
>
> ```python
> import gumbi as gmb
> import seaborn as sns
> cars = sns.load_dataset('mpg').dropna()
> ds = gmb.DataSet(cars, outputs=['mpg', 'acceleration'], log_vars=['mpg', 'acceleration', 'weight', 'horsepower', 'displacement'])
>
> ```
>
> Create a Gumbi `GP` object and fit a model that predicts _mpg_ from _horsepower_:
>
> ```python
> gp = gmb.GP(ds)
> gp.fit(outputs=['mpg'], continuous_dims=['horsepower']);
>
> ```
>
> Make predictions and plot!
>
> ```python
> X = gp.prepare_grid()
> y = gp.predict_grid()
> gmb.ParrayPlotter(X, y).plot()
> sns.scatterplot(data=cars, x='horsepower', y='mpg', color=sns.cubehelix_palette()[-1], alpha=0.5);
>
> ```
>
> 
Gumbi can handle multiple continuous input dimensions, multiple categorical input dimensions (correlated via a Linear Model of Coregionalization), and multiple outputs (also via LMC), along with Normal, Log-Normal, and Logit-Normal variables. Right now, only Marginal and MarginalSparse (DTC) implementations are supported, but I hope to add Latent variable support to allow for classification, heteroskedasticity, and other likelihood structures.
There’s still a lot of room for improvement, but the basic structure and functionality is in place. It’s available on [PyPI](https://pypi.org/project/gumbi/), and I’ll try to put it on conda-forge soon. Take a look at the [docs](https://johngoertz.github.io/Gumbi/index.html), which has thorough API documentation and example notebooks. I plan to add more explanations of some of the underlying data structures, and please let me know if anything is unclear!
I’m still developing the package, and contributions are welcome! This is my first open-source project, so bear with me while I figure it out as I go. I’m very open to suggestions on everything from API structure to inner workings and git guidelines. There are many features I still plan to implement, some of them obvious, so please open issues requesting specific features as this will help me prioritize.
And finally, let me know what you make with it! I’m excited to see how it does out “in the wild”.
---
### Author:  [@zweli](https://discourse.pymc.io/u/zweli)
#### Post date: [December 2, 2021, 8:37pm UTC](https://discourse.pymc.io/t/introducing-gumbi-the-gaussian-process-model-building-interface/8377/2 "2021-12-02T20:37:22Z")
Well done!
---
### Author:  [@sree\_datta](https://discourse.pymc.io/u/sree_datta)
#### Post date: [February 11, 2022, 3:29am UTC](https://discourse.pymc.io/t/introducing-gumbi-the-gaussian-process-model-building-interface/8377/3 "2022-02-11T03:29:46Z")
@BioGoertz this is a fabulous addition to the Pymc3 interfaces. I use `Bambi` quite frequently to mimic the interface I have in R called `brms`, a front-end for `RStan`.
I will use your package and report back on its use here and at Github. Thank you for creating and sharing this wonderful tool that makes GP model building easier!
---
### Author:  [@Ritvik\_Sahajpal](https://discourse.pymc.io/u/Ritvik_Sahajpal)
#### Post date: [February 11, 2022, 4:18pm UTC](https://discourse.pymc.io/t/introducing-gumbi-the-gaussian-process-model-building-interface/8377/4 "2022-02-11T16:18:33Z")
@BioGoertz, thanks for a great package! I look forward to using it. Would it be possible to have an example for folks who are coming from using gradient boosting trees or similar models to use Gumbi instead? I can help provide a gradient boosting tree based example, but not sure how to best represent it using Gumbi.
---
### Author:  [@sree\_datta](https://discourse.pymc.io/u/sree_datta)
#### Post date: [February 14, 2022, 6:25am UTC](https://discourse.pymc.io/t/introducing-gumbi-the-gaussian-process-model-building-interface/8377/5 "2022-02-14T06:25:52Z")
Hi @BioGoertz I have installed and tested `gumbi` on both `Windows 10 and WSL2` under Anaconda setup. Please note that my `numpy` version was `1.21.5`. I have tried but unable to install a numpy version \>=1.15 #\<= 1.19.3.
I installed an environment where numpy version is 1.19.3, but when I install other packages, numpy gets upgraded to 1.21.5.
Here are the commands I ran and the errors I got (Any resolution would be very helpful):
````nohighlight
import seaborn as sns
cars = sns.load_dataset('mpg').dropna()
ds = gmb.DataSet(cars, outputs=['mpg', 'acceleration'], log_vars=['mpg', 'acceleration', 'weight', 'horsepower', 'displacement'])```
````
C:\Users\sreedatta\Anaconda3\envs\gumbi\_env\lib\site-packages\theano\tensor\elemwise.py:826: RuntimeWarning: invalid value encountered in log
variables = ufunc(\*ufunc\_args, \*\*ufunc\_kwargs)
```auto
```
X = gp.prepare\_grid()
y = gp.predict\_grid()
gmb.ParrayPlotter(X, y).plot()
sns.scatterplot(data=cars, x=‘horsepower’, y=‘mpg’, color=sns.cubehelix\_palette)
```auto
```
TypeError Traceback (most recent call last)
Input In [5], in
2 y = gp.predict\_grid()
3 gmb.ParrayPlotter(X, y).plot()
----\> 4 sns.scatterplot(data=cars, x=‘horsepower’, y=‘mpg’, color=sns.cubehelix\_palette)
File ~\Anaconda3\envs\gumbi\_env\lib\site-packages\seaborn\_decorators.py:46, in \_deprecate\_positional\_args..inner\_f(\*args, \*\*kwargs)
36 warnings.warn(
37 "Pass the following variable{} as {}keyword arg{}: {}. "
38 "From version 0.12, the only valid positional argument "
(…)
43 FutureWarning
44 )
45 kwargs.update({k: arg for k, arg in zip(sig.parameters, args)})
—\> 46 return f(\*\*kwargs)
File ~\Anaconda3\envs\gumbi\_env\lib\site-packages\seaborn\relational.py:827, in scatterplot(x, y, hue, style, size, data, palette, hue\_order, hue\_norm, sizes, size\_order, size\_norm, markers, style\_order, x\_bins, y\_bins, units, estimator, ci, n\_boot, alpha, x\_jitter, y\_jitter, legend, ax, \*\*kwargs)
823 return ax
825 p.\_attach(ax)
→ 827 p.plot(ax, kwargs)
829 return ax
File ~\Anaconda3\envs\gumbi\_env\lib\site-packages\seaborn\relational.py:608, in \_ScatterPlotter.plot(self, ax, kws)
603 scout\_size = max(
604 np.atleast\_1d(kws.get(“s”, [])).shape[0],
605 np.atleast\_1d(kws.get(“c”, [])).shape[0],
606 )
607 scout\_x = scout\_y = np.full(scout\_size, np.nan)
→ 608 scout = ax.scatter(scout\_x, scout\_y, \*\*kws)
609 s = kws.pop(“s”, scout.get\_sizes())
610 c = kws.pop(“c”, scout.get\_facecolors())
File ~\Anaconda3\envs\gumbi\_env\lib\site-packages\matplotlib\__init_\_.py:1412, in \_preprocess\_data..inner(ax, data, \*args, \*\*kwargs)
1409 @functools.wraps(func)
1410 def inner(ax, \*args, data=None, \*\*kwargs):
1411 if data is None:
→ 1412 return func(ax, \*map(sanitize\_sequence, args), \*\*kwargs)
1414 bound = new\_sig.bind(ax, \*args, \*\*kwargs)
1415 auto\_label = (bound.arguments.get(label\_namer)
1416 or bound.kwargs.get(label\_namer))
File ~\Anaconda3\envs\gumbi\_env\lib\site-packages\matplotlib\axes\_axes.py:4387, in Axes.scatter(self, x, y, s, c, marker, cmap, norm, vmin, vmax, alpha, linewidths, edgecolors, plotnonfinite, \*\*kwargs)
4384 if edgecolors is None:
4385 orig\_edgecolor = kwargs.get(‘edgecolor’, None)
4386 c, colors, edgecolors =
→ 4387 self.\_parse\_scatter\_color\_args(
4388 c, edgecolors, kwargs, x.size,
4389 get\_next\_color\_func=self.\_get\_patches\_for\_fill.get\_next\_color)
4391 if plotnonfinite and colors is None:
4392 c = np.ma.masked\_invalid(c)
File ~\Anaconda3\envs\gumbi\_env\lib\site-packages\matplotlib\axes\_axes.py:4160, in Axes.\_parse\_scatter\_color\_args(c, edgecolors, kwargs, xsize, get\_next\_color\_func)
4158 if kwcolor is not None:
4159 try:
→ 4160 mcolors.to\_rgba\_array(kwcolor)
4161 except ValueError as err:
4162 raise ValueError(
4163 "‘color’ kwarg must be a color or sequence of color "
4164 "specs. For a sequence of values to be color-mapped, use "
4165 “the ‘c’ argument instead.”) from err
File ~\Anaconda3\envs\gumbi\_env\lib\site-packages\matplotlib\colors.py:363, in to\_rgba\_array(c, alpha)
358 if isinstance(c, str):
359 raise ValueError("Using a string of single character colors as "
360 "a color sequence is not supported. The colors can "
361 “be passed as an explicit list instead.”)
→ 363 if len(c) == 0:
364 return np.zeros((0, 4), float)
366 # Quick path if the whole sequence can be directly converted to a numpy
367 # array in one shot.
TypeError: object of type ‘function’ has no len()
```auto
```
---
### Author:  [@BioGoertz](https://discourse.pymc.io/u/BioGoertz)
#### Post date: [February 14, 2022, 9:27am UTC](https://discourse.pymc.io/t/introducing-gumbi-the-gaussian-process-model-building-interface/8377/6 "2022-02-14T09:27:20Z")
Hi sree\_datta, thanks for giving it a shot! That error is just down to the `color` argument to seaborn’s `scatterplot`. You provided `color=sns.cubehelix_palette`, which is a function, but you need to provide it with a single color (in the example I specify `color=sns.cubehelix_palette()[-1]`, which gives a dark purple, but you’re of course free to use any color seaborn accepts).
The numpy issue is unrelated, and should only cause an issue if you need the MultivariateUncertainParameterArray. Higher versions of numpy were giving really bizarre low-level linear algebra errors when calling `mvup.dist` and `mvup.cov`. In fact, if you wanted to try running the section on [Multi-output regression](https://johngoertz.github.io/Gumbi/notebooks/examples/Cars_Dataset.html#Multi-output-regression) (specifically the multivariate sampling bit) in the Cars Dataset notebook with your setup, and let me know if you get weird errors, that would help a lot!
---
### Author:  [@BioGoertz](https://discourse.pymc.io/u/BioGoertz)
#### Post date: [February 14, 2022, 9:41am UTC](https://discourse.pymc.io/t/introducing-gumbi-the-gaussian-process-model-building-interface/8377/7 "2022-02-14T09:41:38Z")
Thanks for your interest! I’m really not at all familiar with gradient boosting trees, could you provide a rough example of your workflow? Take a look at the [Cars Dataset notebook](https://johngoertz.github.io/Gumbi/notebooks/examples/Cars_Dataset.html) and see if that helps clarify its use.
I’m guessing you’ll need to be a bit more careful with your features using a GP compared to a GBT. In Gumbi, you specify which of your variables need to be log- or logit-transformed when you create a `gmb.DataSet` from a `pd.Dataframe`. Then, when you go to build/fit the GP, you declare which inputs you’re actually interested in by specifying which are to be treated as continuous variables (`continuous_dims`) and which as categorical (`categorical_dims`).
Continuous variables are treated as you might expect, while categorical variables are used to define separate but _correlated_ Gaussian Processes. If you have one categorical variable, each value in that column defines a separate GP; if you have multiple, each unique combination of categorical variables gets its own. This is done through what is known as the “Linear Model of Corregionalization”, or LMC. As this name suggests, the GP learns a _linear_ correlation between the classes, but this can be positive or negative, and it will happily learn there’s no correlation if the data indicate that.
Finally, multi-output regression is also acheived through an LMC implementation.
Hope that helps!
---
### Author:  [@sree\_datta](https://discourse.pymc.io/u/sree_datta)
#### Post date: [February 14, 2022, 3:32pm UTC](https://discourse.pymc.io/t/introducing-gumbi-the-gaussian-process-model-building-interface/8377/8 "2022-02-14T15:32:53Z")
Thanks @BioGoertz for your quick feedback. I will make the changes pertinent to `seaborn` and also test the Multi-output regression example. I will share the test results back here.
---
### Author:  [@BioGoertz](https://discourse.pymc.io/u/BioGoertz)
#### Post date: [February 14, 2022, 3:54pm UTC](https://discourse.pymc.io/t/introducing-gumbi-the-gaussian-process-model-building-interface/8377/9 "2022-02-14T15:54:43Z")
Great! If you could actually log any issues/errors like this on the [Github page](https://github.com/JohnGoertz/Gumbi/issues) that’ll make them easier to keep track of, and we can keep this thread for more high-level, conceptual discussion.
---
### Author:  [@sree\_datta](https://discourse.pymc.io/u/sree_datta)
#### Post date: [February 15, 2022, 11:35pm UTC](https://discourse.pymc.io/t/introducing-gumbi-the-gaussian-process-model-building-interface/8377/10 "2022-02-15T23:35:21Z")
@BioGoertz I have opened an issue ticket here [Assertion Error with Multi Output Regression on WSL2](https://github.com/JohnGoertz/Gumbi/issues/15)
---
### Author:  [@sree\_datta](https://discourse.pymc.io/u/sree_datta)
#### Post date: [February 16, 2022, 5:18pm UTC](https://discourse.pymc.io/t/introducing-gumbi-the-gaussian-process-model-building-interface/8377/11 "2022-02-16T17:18:31Z")
@BioGoertz unable to install `gumbi-0.1.8.` using `pip install gumbi`. I have posted the error at GitHub
---
### Author:  [@BioGoertz](https://discourse.pymc.io/u/BioGoertz)
#### Post date: [February 18, 2022, 9:13am UTC](https://discourse.pymc.io/t/introducing-gumbi-the-gaussian-process-model-building-interface/8377/12 "2022-02-18T09:13:50Z")
(this is now fixed - it installs now!)
---
### Author:  [@Oliver](https://discourse.pymc.io/u/Oliver)
#### Post date: [October 3, 2022, 3:49pm UTC](https://discourse.pymc.io/t/introducing-gumbi-the-gaussian-process-model-building-interface/8377/13 "2022-10-03T15:49:59Z")
Hi there, cool new package! I have had fun with bambi and thought I’d try this over vanilla solutions. To be honest I am a bit confused though, I just did a simple model with pymc3 and now tried gumbi. But the predictions with predict are way off compared to MarginalApprox (MarginalSparse in v3). What exactly does prepare grid do when you have multiple continuous parameters?
I.e. I am quite confused as to why the ParameterArray input of points has to be 1d.
Sorry for the thousand probably silly newbie questions.
---
### Author:  [@BioGoertz](https://discourse.pymc.io/u/BioGoertz)
#### Post date: [November 15, 2022, 3:56pm UTC](https://discourse.pymc.io/t/introducing-gumbi-the-gaussian-process-model-building-interface/8377/14 "2022-11-15T15:56:08Z")
Hi @Oliver, thanks for giving it a shot! Sorry for the delay in getting back to you. That’s interesting that the predictions are very different, could you provide a minimal example and [open an issue on Github](https://github.com/JohnGoertz/Gumbi/issues)?
To your question about [`prepare_grid`](https://github.com/JohnGoertz/Gumbi/blob/main/gumbi/regression/base.py#L585), it just builds a Cartesian grid across all dimensions. It defaults to 100 points spanning -2.5σ to +2.5σ, where σ is the standard deviation of the (transformed) dimension. I’m not sure what you mean by _“the ParameterArray input of points has to be 1d”_. Maybe you’re referring to the `at` keyword argument? That’s intended to indicate a specific “slice” of your multidimensional space. You can see its usage in [this example](https://johngoertz.github.io/Gumbi/notebooks/examples/Cars_Dataset.html#3D-regression-with-2D-predictions) in the documentation:
```python
...
gp.fit(outputs=['mpg'], continuous_dims=['horsepower','weight', 'cylinders'], linear_dims=['horsepower','weight', 'cylinders']);
for ax, cyl in zip(axs, cylinders):
XY = gp.prepare_grid(at=gp.parray(cylinders=cyl))
X = XY['horsepower']
Y = XY['weight']
z = gp.predict_grid(with_noise=False)
...
```
There, a GP is fit on the “cars” dataset. We’re interested in looking at the dependence of _mpg_ on _horsepower_ and _weight_ surface at each value of _cylinder_. Now, the _cylinder_ dimension is technically discrete, not continuous, but I haven’t implemented a discrete kernel yet, so it’s treated as continuous. For plotting purposes, we use the `at` keyword argument to specify the specific “slice” along the _cylinder_ dimension in _cylinder_-_horsepower_-_weight_ space to make predictions along.
Hope that helps, let me know if anything’s unclear! I am still developing/maintaining the package, as I have time, so suggestions/issues/questions are very welcome!