Questions tagged [bayesian-optimization]

For questions related to Bayesian optimization (BO), which is a technique used to model an unknown function (that is expensive to evaluate), based on concepts of a surrogate model (which is usually a Gaussian process, which models the unknown function), Bayesian inference (to update the Gaussian process) and an acquisition function (which guides the Bayesian inference). BO can be used for hyper-parameter optimization.

Filter by
Sorted by
Tagged with
1 vote
0 answers
15 views

Why does importance sampling work with latent variable models?

Caveat: importance sampling doesn't actually work for variational auto-encoders, but the question makes sense regardless In "L4 Latent Variable Models (VAE) -- CS294-158-SP20 Deep Unsupervised ...
user avatar
  • 141
1 vote
0 answers
19 views

Alternatives to Bayesian optimization

I am given a dataset $\mathcal{D} = \{\mathbf{x}_i\}_{i=1}^n$ and I need to find the point (in my case a material) $\mathbf{x}^*$ that maximizes a property $y$ (which can be obtained from a black-box ...
user avatar
  • 121
1 vote
0 answers
9 views

Bayesian optimization with confidence bound not working

I have a simple MLP for which I want to optimize some hyperparameters. I have fixed the number of hidden layers (for unrelated reasons) to be 3. So the hyperparameters being optimized through Bayesian ...
user avatar
  • 111
1 vote
1 answer
41 views

How can I interpret the value returned by score(X) method of sklearn.neighbors.KernelDensity?

For sklearn.neighbors.KernelDensity, its score(X) method according to the sklearn KDE documentation says: Compute the log-...
user avatar
  • 195
0 votes
0 answers
38 views

How to identify clusters of best hyperparameters from noisy hyperparameter optimization?

I'd like to optimize the hyperparameters for a random forest model that is already somewhat time-consuming to train. With adding cross-validation and multiple hyperparameter combinations, I might be ...
user avatar
4 votes
1 answer
101 views

Bayesian hyperparameter optimization, is it worth it?

In the Deep Learning book by Goodfellow et al., section 11.4.5 (p. 438), the following claims can be found: Currently, we cannot unambiguously recommend Bayesian hyperparameter optimization as an ...
user avatar
3 votes
0 answers
83 views

Can we use a Gaussian process to approximate the belief distribution at every instant in a POMDP?

Suppose $x_{t+1} \sim \mathbb{P}(\cdot | x_t, a_t)$ denotes the state transition dynamics in a reinforcement learning (RL) problem. Let $y_{t+1} = \mathbb{P}(\cdot | x_{t+1})$ denote the noisy ...
user avatar
4 votes
0 answers
269 views

How can I draw a Bayesian network for this problem with birds?

I am working on the following problem to gain an understanding of Bayesian networks and I need help drawing it: Birds frequently appear in the tree outside of your window in the morning and ...
user avatar
  • 41
1 vote
1 answer
104 views

Understanding Bayesian Optimisation graph

I came across the concept of Bayesian Occam Razor in the book Machine Learning: a Probabilistic Perspective. According to the book: Another way to understand the Bayesian Occam’s razor effect is ...
user avatar
2 votes
0 answers
25 views

Is it normal to see oscillations in tested hyperparameters during bayesian optimisation?

I've been trying out bayesian hyperparameter optimisation (with TPE) on a simple CNN applied to the MNIST handwritten digit dataset. I noticed that over iterations of the optimisation loop, the tested ...
user avatar
6 votes
3 answers
2k views

What is a "surrogate model"?

In the following paragraph from the book Automated Machine Learning: Methods, Systems, Challenges (by Frank Hutter et al.) In this section we first give a brief introduction to Bayesian ...
user avatar
1 vote
1 answer
168 views

Are there Python packages for recent Bayesian optimization methods? [closed]

I want to try and compare different optimization methods in some datasets. I know that in scikit-learn there are some corresponding functions for the grid and random search optimizations. However, I ...
user avatar