PyMC for Bayesian Optimization

Hi Guys,

Have anyone have used PyMC for Bayesian Optimization? If so, are there any documents or tutorial I could look at?

Thanks in advance!

1 Like

You can check out this blog post from @twiecki.

Thanks @cluhmann.

Just to clarify though, Bayesian Optimization usually refers to trying to optimize an objective function using GPs, which this blog post does not address. I don’t think we have any materials on that, but it should be pretty straight-forward to with our GP capabilities.


Thank you twiecki for the information. What I am actually trying to do is to use Bayesian NN instead of GP to perform Baysian optimization due to the nature of my data is not compatible with GP. So I am wondering if you know any sources talking about it? Or have you and other tried using PyMC with some Bayesian OPT platforms such as Ax or Botorch? Thank you!

There’s a few papers that do that, but I don’t know of any software that does it. If you’re using a NN instead of a GP, however, I’m not sure you need a Bayesian NN, in which case you wouldn’t get much benefit from using PyMC.

I’m looking to do something similar, neither using GPs nor Bayesian NNs, and struggle to find an elegant solution. I have a hierarchical forward model of controlled_variables → latent_variables _modulated_by_observed_covariates → observed_outcome in pymc trained using HMC and I want to optimize controlled_variables using some acquisition function, say Expected Improvement. Pymc/ Aesara should have everything I need for that under the hood, but I wondered whether I am missing a simple trick to make best use of it with a defined model and a trace of MCMC samples.