Questions tagged [bayesian-optimization]

For questions related to Bayesian optimization (BO), which is a technique used to model an unknown function (that is expensive to evaluate), based on concepts of a surrogate model (which is usually a Gaussian process, which models the unknown function), Bayesian inference (to update the Gaussian process) and an acquisition function (which guides the Bayesian inference). BO can be used for hyper-parameter optimization.

Filter by
Sorted by
Tagged with
0 votes
0 answers
43 views

Optimizing Stop Loss Percentage for a Specific Model Based on Stock Price to Maximize Expected Value

I'm fine-tuning a specific trading model, and a crucial parameter I'm keen on optimizing is the stop loss percentage. The primary objective is to maximize the Expected Value (EV), formulated as: $$EV =...
David's user avatar
  • 11
2 votes
1 answer
178 views

In which community does using a Bayesian regression model as a reward function with exploration vs. exploitation challenges fall under?

I am trying to find research papers addressing a problem that, in my opinion, deserves significant attention. However, I am having difficulty locating relevant information. To illustrate the problem ...
paul's user avatar
  • 35
0 votes
0 answers
39 views

Very high dimensional optimization with large budget, requiring high quality solutions

What would be theoretically the best performing optimization algorithm(s) in this case? Very high dimensional problem: 250-500 parameters Goal is to obtain very high quality solutions, not just "...
Charly Empereur-mot's user avatar
1 vote
0 answers
10 views

variational inference but with a weighted loglikelihood

I would like to know if it's correct if I substitute in the ELBO formula a weighted sum of the loglikelihood $$\sum E_{q_{\theta}(w)}[w_i \ln{p(y_i|f^{w}(x_i))}]$$ in place of the traditional sum. ...
Alucard's user avatar
  • 111
2 votes
0 answers
16 views

Minimum sampling for maximising the prediction accuracy

Suppose that I'm training a machine learning model to predict people's age by a picture of their faces. Lets say that I have a dataset of people from 1 year olds to 100 year olds. But I want to choose ...
noone's user avatar
  • 123
1 vote
0 answers
83 views

Why does importance sampling work with latent variable models?

Caveat: importance sampling doesn't actually work for variational auto-encoders, but the question makes sense regardless In "L4 Latent Variable Models (VAE) -- CS294-158-SP20 Deep Unsupervised ...
Foobar's user avatar
  • 151
1 vote
0 answers
241 views

Alternatives to Bayesian optimization

I am given a dataset $\mathcal{D} = \{\mathbf{x}_i\}_{i=1}^n$ and I need to find the point (in my case a material) $\mathbf{x}^*$ that maximizes a property $y$ (which can be obtained from a black-box ...
ado sar's user avatar
  • 150
1 vote
0 answers
25 views

Bayesian optimization with confidence bound not working

I have a simple MLP for which I want to optimize some hyperparameters. I have fixed the number of hidden layers (for unrelated reasons) to be 3. So the hyperparameters being optimized through Bayesian ...
NeuroEng's user avatar
  • 121
1 vote
1 answer
186 views

How can I interpret the value returned by score(X) method of sklearn.neighbors.KernelDensity?

For sklearn.neighbors.KernelDensity, its score(X) method according to the sklearn KDE documentation says: Compute the log-...
Arun's user avatar
  • 225
4 votes
1 answer
138 views

Bayesian hyperparameter optimization, is it worth it?

In the Deep Learning book by Goodfellow et al., section 11.4.5 (p. 438), the following claims can be found: Currently, we cannot unambiguously recommend Bayesian hyperparameter optimization as an ...
Stefano Barone's user avatar
3 votes
0 answers
93 views

Can we use a Gaussian process to approximate the belief distribution at every instant in a POMDP?

Suppose $x_{t+1} \sim \mathbb{P}(\cdot | x_t, a_t)$ denotes the state transition dynamics in a reinforcement learning (RL) problem. Let $y_{t+1} = \mathbb{P}(\cdot | x_{t+1})$ denote the noisy ...
math_phile's user avatar
4 votes
0 answers
288 views

How can I draw a Bayesian network for this problem with birds?

I am working on the following problem to gain an understanding of Bayesian networks and I need help drawing it: Birds frequently appear in the tree outside of your window in the morning and ...
Jake's user avatar
  • 41
1 vote
1 answer
130 views

Understanding Bayesian Optimisation graph

I came across the concept of Bayesian Occam Razor in the book Machine Learning: a Probabilistic Perspective. According to the book: Another way to understand the Bayesian Occam’s razor effect is ...
user avatar
2 votes
0 answers
27 views

Is it normal to see oscillations in tested hyperparameters during bayesian optimisation?

I've been trying out bayesian hyperparameter optimisation (with TPE) on a simple CNN applied to the MNIST handwritten digit dataset. I noticed that over iterations of the optimisation loop, the tested ...
Alexander Soare's user avatar
6 votes
3 answers
3k views

What is a "surrogate model"?

In the following paragraph from the book Automated Machine Learning: Methods, Systems, Challenges (by Frank Hutter et al.) In this section we first give a brief introduction to Bayesian ...
yousef yegane's user avatar
1 vote
1 answer
349 views

Are there Python packages for recent Bayesian optimization methods? [closed]

I want to try and compare different optimization methods in some datasets. I know that in scikit-learn there are some corresponding functions for the grid and random search optimizations. However, I ...
Enes Altuncu's user avatar