4
What is Bayesian optimization?
Introduction
Bayesian optimization (BO) is an optimization technique used to model an unknown (usually continuous) function $f: \mathbb{R}^d \rightarrow Y$, where typically $d \leq 20$, so it can be used to solve regression and classification problems, where you want to find an approximation of $f$. In this sense, BO is ...
4
A surrogate model is a simplified model. It is a mapping $y_S=f_S(x)$ that approximates the original model $y=f(x)$, in a given domain, reasonably well. Source: Engineering Design via Surrogate Modelling: A Practical Guide
In the context of Bayesian optimization, one wants to optimize a function $y=f(x)$ which is expensive (very time consuming) to evaluate, ...
2
Efficiently integrating HPO frameworks into an existing project is non-trivial. Most common datasets/tasks already have established architectures/hyperparameters/etc. and require only a few additional tuning parameters. In this case, the benefits (assuming they exist) brought by Bayesian HPO techniques lack behind development time (simplicity), and this is ...
1
Welcome to AI.SE Enes.
I think by random search, you are referring to so-called "black-box optimization". Random search is sometimes used as a name for this, but BBO is a more common name, and might be easier to search for.
There are many BBO techniques. 'random search' is usually used to refer to a hill-climbing algorithm where you start at a random ...
1
Apart from the Scikit-Optimize package related to Scikit-Learn, following are some of the packages related to Bayesian optimization:
GPyOpt
pyGPGO
Hyperopt
bayesian-optimization
safeopt
RoBO
Only top voted, non community-wiki answers of a minimum length are eligible
