Using Bayesian Optimization to suggest next point to evaluate

[denzdelvillar]

Hi,

I am trying to see whether I can use BoTorch for my project. I have a dataset consisting of 8000+ points, each with 20 parameters and a single objective value that I want to minimize. The objective value is obtained by feeding the 20 parameters into a simulation software and post-processing its data output. I want to create a script that uses Bayesian Optimization to output the next set of 20 parameters. I will then feed this into the simulation software and get the objective value. Then I will feed this new data point into the bayesian optimization script so that it will improve the model and suggest me the next set of parameters.

Most of the tutorials generate the initial data points by randomly sampling from the search space and then using an evaluate function. Is there a method I could use to somehow "register" an existing dataset?

Thank you very much!

[Balandat]

Certainly. Instead of assigning random data points you just use the existing data you have. E.g. in this tutorial you'd assign your existing data points to train_x_ei and existing observations to train_obj_ei at the start. That should be it.

That said, 8000+ data points and 20 dimensions are a good amount for the standard GP models that are used in all of the tutorials. The dimensionality means that standard models may not fit well and have very high uncertainty in between data points (resulting in excessive exploration), and the data size will likely cause computational scalability issues.

Before getting into suggesting alternative models (e.g. running an exact GP on a GPU using KeOPS, using an approximate GP such as SKIP, or a variational GP) I'm curious how long a simulation takes and how many more evaluations you are planning to run.

[denzdelvillar]

Thanks for your reply!

It takes about a minute or two to finish a simulation run and extract the objective score from it. The existing data set was itself obtained from an optimization run using evolutionary algorithm from which some pretty good results were already observed. The Bayesian Optimization would build on top of this and try to see whether any further improvements can be observed.

As of the moment there is no definite plan on how many more evaluations to run, but my primary concern is indeed scalability which you have pointed out. I have tried two other Bayesian Optimization libraries before but the computation time was not ideal for me.