Podcast #128: We chat with Kent C Dodds about why he loves React and discuss what life was like in the dark days before Git. Listen now.

Questions tagged [hyperparameter-optimization]

For questions related to the concept of hyper-parameter optimization, that is, the task of finding the best hyper-parameters for a particular learning algorithm (e.g. gradient descent) or model (e.g. a multi-layer neural network) using an optimization method (e.g. Bayesian optimization or genetic algorithms).

Filter by
Sorted by
Tagged with
3
votes
0answers
13 views

How to organize model training hyperparameters

I am working on multiple deep learning projects, most of them in the area of computer vision. For many of them I create multiple models, try different approaches, use various model architectures. And ...
3
votes
2answers
121 views

What is a “surrogate model”?

In the following paragraph from the book Automated Machine Learning: Methods, Systems, Challenges (by Frank Hutter et al.) In this section we first give a brief introduction to Bayesian ...
4
votes
1answer
28 views

How can I avoid overfitting when doing parameter tuning?

I very often applied a grid search to tune the parameters of my supervised model. I have the feeling that parameter tuning will eventually (very often) lead to overfitting? Is this crazy to say? Is ...
1
vote
1answer
33 views

How to use a deep learning network on new data-set?

I am trying to use a network for classification. This network works very well on the author's example data, but doesn't work on new data. Currently, I am using the popular EEG Motor Movement/Imagery ...
0
votes
0answers
14 views

Choosing best combinations from all possible combination expressions based few variables, unary operators, binary operators

I have a few financial variables of a stock universe like OHLC prices, volume, and other fundamentals with varying time-frequency. Using this set I'm creating an expression that gives the weights to ...
5
votes
1answer
37 views

How can we conclude that an optimization algorithm is better than another one

When we test a new optimization algorithm, what the process that we need to do?For example, do we need to run the algorithm several times, and pick a best performance,i.e., in terms of accuracy, f1 ...
1
vote
0answers
35 views

Threshold selection for Siamese network hyper-parameter tuning

I'm interested in modeling a Siamese network for facial verification. I've already written a simple working model that inputs feature vectors generated from two CNNs with shared weights then outputs a ...
0
votes
0answers
18 views

Improving Recall of a Certain Class

Let's say that we have a test data set with $20,000$ observations for which we want to make a binary prediction for. When we apply our best trained model to this data set (e.g. logistic regression ...
4
votes
1answer
78 views

An intuitive explanation of Adagrad, its purpose and its formula

It (Adagrad) adapts the learning rate to the parameters, performing smaller updates (i.e. low learning rates) for parameters associated with frequently occurring features, and larger updates (i.e. ...
4
votes
1answer
89 views

What is the best measure against overfitting?

I wanted to ask about the methodology of testing the ML models against overfitting. Please note that I don't mean any overfitting reducing methods like regularisation, just a measure to judge whether ...
1
vote
0answers
27 views

How do you scale your ML problems?

While I have limited resource usually to train my machine learning models, I often find that my hyperparameter optimization procedure is not necessary using all my GPU and CPU, and that is because the ...
1
vote
2answers
37 views

Does this hyperparameter optimisation approach yield the optimal hyperparameters?

Say I have a ML model which is not very costly to train. It has around say 5 hyperparameters. One way to select best hyperparameters would be to keep all the other hyperparamaters fixed and train ...
2
votes
1answer
66 views

Can we automate the choice of the hyper-parameters of the evolutionary algorithms?

Certain hyper-parameters (e.g. the size of the offspring generation or the definition of the fitness function) and the design (e.g. how the mutation is performed) of evolutionary algorithms usually ...
0
votes
0answers
45 views

Why doesnt my lstm model for time series prediction improve after certain level of performance?

I created an lstm model which predicts multioutput sequeances. It takes variable length sequences as input. These sequences are padded with zero to obtain equal length. Note that the time series are ...
0
votes
1answer
36 views

Evolving Machine Learning

It seems to me that, right now, the key to making a good Machine Learning model is in choosing the right combination of hyper-parameters. Firstly: Am I right in saying, if a model is able to tune it'...
0
votes
0answers
27 views

How to use TensorFlow with hyperparameter tuning to optimize parameters for a robot simulator

I am trying to implement a DNN to optimize a set of 7 parameters that are used in a robot swarm simulator on the ARGoS platform. the program is a compiled C++ executable that reads the parameters from ...
5
votes
3answers
128 views

How is neural architecture search performed?

I have come across something that IBM offers called neural Architecture search. You feed it a data set and it outputs an initial neural Architecture that you can train. How is neural architecture ...