PyMC for Bayesian Optimization

Questions version agnostic
1

Hi Guys,

Have anyone have used PyMC for Bayesian Optimization? If so, are there any documents or tutorial I could look at?

Thanks in advance!

1 Like
2

You can check out this blog post from @twiecki.

3

Thanks @cluhmann.

Just to clarify though, Bayesian Optimization usually refers to trying to optimize an objective function using GPs, which this blog post does not address. I don’t think we have any materials on that, but it should be pretty straight-forward to with our GP capabilities.

2 Likes
4

Thank you twiecki for the information. What I am actually trying to do is to use Bayesian NN instead of GP to perform Baysian optimization due to the nature of my data is not compatible with GP. So I am wondering if you know any sources talking about it? Or have you and other tried using PyMC with some Bayesian OPT platforms such as Ax or Botorch? Thank you!

5

There’s a few papers that do that, but I don’t know of any software that does it. If you’re using a NN instead of a GP, however, I’m not sure you need a Bayesian NN, in which case you wouldn’t get much benefit from using PyMC.

6

I’m looking to do something similar, neither using GPs nor Bayesian NNs, and struggle to find an elegant solution. I have a hierarchical forward model of controlled_variables → latent_variables _modulated_by_observed_covariates → observed_outcome in pymc trained using HMC and I want to optimize controlled_variables using some acquisition function, say Expected Improvement. Pymc/ Aesara should have everything I need for that under the hood, but I wondered whether I am missing a simple trick to make best use of it with a defined model and a trace of MCMC samples.