4

What is Bayesian optimization? Introduction Bayesian optimization (BO) is an optimization technique used to model an unknown (usually continuous) function $f: \mathbb{R}^d \rightarrow Y$, where typically $d \leq 20$, so it can be used to solve regression and classification problems, where you want to find an approximation of $f$. In this sense, BO is ...


4

A surrogate model is a simplified model. It is a mapping $y_S=f_S(x)$ that approximates the original model $y=f(x)$, in a given domain, reasonably well. Source: Engineering Design via Surrogate Modelling: A Practical Guide In the context of Bayesian optimization, one wants to optimize a function $y=f(x)$ which is expensive (very time consuming) to evaluate, ...


2

Efficiently integrating HPO frameworks into an existing project is non-trivial. Most common datasets/tasks already have established architectures/hyperparameters/etc. and require only a few additional tuning parameters. In this case, the benefits (assuming they exist) brought by Bayesian HPO techniques lack behind development time (simplicity), and this is ...


1

Welcome to AI.SE Enes. I think by random search, you are referring to so-called "black-box optimization". Random search is sometimes used as a name for this, but BBO is a more common name, and might be easier to search for. There are many BBO techniques. 'random search' is usually used to refer to a hill-climbing algorithm where you start at a random ...


1

Apart from the Scikit-Optimize package related to Scikit-Learn, following are some of the packages related to Bayesian optimization: GPyOpt pyGPGO Hyperopt bayesian-optimization safeopt RoBO


Only top voted, non community-wiki answers of a minimum length are eligible