Gradients for surrogate models in pymc3

Questions
1

I’m using Bayesian analysis to inter the parameters of models for composite materials. The models are multilevel, but deterministic and very expensive to evaluate so we use surrogate functions for stochastic analysis.

I’m trying to figure out how to get the gradient for the surrogate functions in theano so I can run NUTS on a few of these problems, but I’m totally stuck.

I attached a toy example where the goal is to infer unknown spring stiffnesses k1 and k2 given observations of the equavialent stiffnesses for the springs in series and in parrallel.

I define the theano op which uses the surrogate function (scipy’s rbf) as follows

rbfKep = Rbf(k1Samples,k2Samples,keqpSamples)

@theano.compile.ops.as_op(itypes=[tt.dscalar, tt.dscalar],otypes=[tt.dscalar])
def evalRbfKes(k1val,k2val):
return rbfKes(k1val,k2val)

As far as I know scipy does not implement a gradient for their RBF, so is there a way theano can compute it?

Springs.py (2.2 KB)

2

You can try the Interpolate class in pymc3, see an example here:
http://docs.pymc.io/notebooks/updating_priors.html

3

I need multivariate interpolation for this application.

I’ll keep working on it. I found some papers on using surrogate models with HMC to generate proposals, but the the proposals themselves are evaluated with the full model. Sounds interesting.