Questions tagged [bayesian-optimization]

For questions related to Bayesian optimization (BO), which is a technique used to model an unknown function (that is expensive to evaluate), based on concepts of a surrogate model (which is usually a Gaussian process, which models the unknown function), Bayesian inference (to update the Gaussian process) and an acquisition function (which guides the Bayesian inference). BO can be used for hyper-parameter optimization.

12 questions
Filter by
Sorted by
Tagged with
1 vote
15 views

Why does importance sampling work with latent variable models?

Caveat: importance sampling doesn't actually work for variational auto-encoders, but the question makes sense regardless In "L4 Latent Variable Models (VAE) -- CS294-158-SP20 Deep Unsupervised ...
• 141
1 vote
19 views

Alternatives to Bayesian optimization

I am given a dataset $\mathcal{D} = \{\mathbf{x}_i\}_{i=1}^n$ and I need to find the point (in my case a material) $\mathbf{x}^*$ that maximizes a property $y$ (which can be obtained from a black-box ...
• 121
1 vote
9 views

Bayesian optimization with confidence bound not working

I have a simple MLP for which I want to optimize some hyperparameters. I have fixed the number of hidden layers (for unrelated reasons) to be 3. So the hyperparameters being optimized through Bayesian ...
• 111
1 vote
41 views

How can I interpret the value returned by score(X) method of sklearn.neighbors.KernelDensity?

For sklearn.neighbors.KernelDensity, its score(X) method according to the sklearn KDE documentation says: Compute the log-...
• 195
38 views

How to identify clusters of best hyperparameters from noisy hyperparameter optimization?

I'd like to optimize the hyperparameters for a random forest model that is already somewhat time-consuming to train. With adding cross-validation and multiple hyperparameter combinations, I might be ...
101 views

Bayesian hyperparameter optimization, is it worth it?

In the Deep Learning book by Goodfellow et al., section 11.4.5 (p. 438), the following claims can be found: Currently, we cannot unambiguously recommend Bayesian hyperparameter optimization as an ...
83 views

Can we use a Gaussian process to approximate the belief distribution at every instant in a POMDP?

Suppose $x_{t+1} \sim \mathbb{P}(\cdot | x_t, a_t)$ denotes the state transition dynamics in a reinforcement learning (RL) problem. Let $y_{t+1} = \mathbb{P}(\cdot | x_{t+1})$ denote the noisy ...
269 views

How can I draw a Bayesian network for this problem with birds?

I am working on the following problem to gain an understanding of Bayesian networks and I need help drawing it: Birds frequently appear in the tree outside of your window in the morning and ...
• 41
1 vote
104 views

Understanding Bayesian Optimisation graph

I came across the concept of Bayesian Occam Razor in the book Machine Learning: a Probabilistic Perspective. According to the book: Another way to understand the Bayesian Occam’s razor effect is ...
25 views

Is it normal to see oscillations in tested hyperparameters during bayesian optimisation?

I've been trying out bayesian hyperparameter optimisation (with TPE) on a simple CNN applied to the MNIST handwritten digit dataset. I noticed that over iterations of the optimisation loop, the tested ...
• 1,139
2k views

What is a "surrogate model"?

In the following paragraph from the book Automated Machine Learning: Methods, Systems, Challenges (by Frank Hutter et al.) In this section we first give a brief introduction to Bayesian ...
1 vote
168 views

Are there Python packages for recent Bayesian optimization methods? [closed]

I want to try and compare different optimization methods in some datasets. I know that in scikit-learn there are some corresponding functions for the grid and random search optimizations. However, I ...
• 153