Multidimensional Sparse GP

Questions
1

Hey! I’m trying to implement a multidimensional sparse GP but I’m unsure about the syntax for passing arguments to the marginal likelihood function. I’ve managed the following:

` with pm.Model() as model:

        ℓ = pm.HalfNormal('ℓ', sigma=5)

        lenght_scale_x = pm.HalfNormal('lenght_scale x', sigma=5)

        lenght_scale_c = pm.HalfNormal('lenght_scale c', sigma=5)

        η = pm.HalfCauchy("η", beta=5)

        cov = η**2 * \

            pm.gp.cov.Exponential(1, ls=lenght_scale_c) * \

            pm.gp.cov.Periodic(1, period=ℓ, ls=lenght_scale_x)

        σ = pm.HalfNormal("σ", sigma=0.1)

        gp = pm.gp.MarginalSparse(cov_func = cov, approx="FITC")

        # Inducing points for the sparse aproximation

        Xu = pm.Uniform("Xu",lower = 0, upper = X_1.max(),

                        shape=(10,))

        C_u = pm.Uniform("C_u",lower = 1e-8, upper = c.max(),

                shape=(10,))

        input_u = tt.concatenate([Xu, C_u], axis = 1)

        y_ = gp.marginal_likelihood("y", X=input_1, Xu=input_u, y=f_true, noise=σ)`

Which gives the following error:
ValueError: Join argument "axis" is out of range (given input dimensions)
I’ve tried transposing the arrays entering input_u and transposing input_u but it doesn’t seem to work that way.
Also, is the cov function correctly defined? I started this using a kronecker covariance and thought it would be usefull to define the cov func this way when trying to implement the sparse approximation.
Any help would be highly appreciated!

2

Could you describe your desired model a little more? At first glance, it looks like your covariance functions don’t have the active_dims argument specified. It appears to me that you desire to have an exponential kernel over the first dimension and a periodic kernel over the second dimension. Is that right?

Also, if you could supply some data, real or simulated so that we could run your example, that would be really helpful!

3

Thanks for the reply! I’m trying to calibrate a FEM model for the Euler-Bernoulli Equation using the framework by Kennedy & O’Hagan (The paper in question). The first step of the process is to obtain the parameters of a GP of the computer model with it’s dimensions being time and various physical parameters. Here I’m only trying to use the viscous dampening parameter as an extra dimension.
Since this problem is bound to scale pourly when increasing the number of dimensions being used I wanted to implement some of the methods used for improving scalling like the sparse approximation.
In the uploaded file is a reproduction of what I want to achieve with the model (using a dummy equation to generate model data)
KOH_Example.py (2.8 KB)
Thanks again!!

4

Bump cause of urgency! :upside_down_face:

5

I’ve never seen a model with latent inducing points before. Is that something that can be handled in PyMC3? @bwengals

6

Not sure if this is the underlying problem, but I tried adding the concatenate to KOH_example.py

    input_1 = [X_1.reshape(-1, 1), c.reshape(-1, 1)]
    input_1 = np.concatenate(input_1, axis=1)

and the model sampled after that without errors (not sure if the traces are OK). Does this help?

1 Like
7

Yes!! This Helps, Thanks!!!

8

Yep you could, the code won’t break. Not sure if it would work well though, there may be a reason it’s not common

9

Just to confirm if I instead want to use the kmeans algorithm I must specify a Xu object for each dimension and then concatenate both using Theano?

10

Yep that would work. I’d definitely try Kmeans here. You won’t need to split by dimension though, Kmeans will work just fine across Xu (barring curse of dimensionality type problems).